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EXECUTIVE  SUMMARY 


The  Navy  and  Marine  Corps  use  the  ESSENCE  system  for  early  event  detection 
of  diseases  and  other  public  health  threats  to  the  United  States  military  and  for  situational 
awareness  on  the  location  and  spread  of  such  diseases.  BUMEDINST  6220. 12B 
mandated  that  “health  surveillance  shall  be  conducted  to  enable  early  intervention  and 
control  strategies.”  Navy  ESSENCE  monitors  are  health  professionals,  including 
preventive  medicine  technicians,  doctors,  nurses,  public  health  professionals,  and 
epidemiologists. 

In  accordance  with  the  Navy  Bureau  of  Medicine  and  Surgery  (BUMED) 
mandate,  the  Navy  and  Marine  Corps  Public  Health  Center  (NMCPHC)  sponsored  a 
survey  of  Navy  and  Marine  Corps  ESSENCE  account  holders  about  their  training, 
employment,  and  perceived  value  of  the  system.  This  survey  differs  from  previous 
ESSENCE  surveys  in  a  number  of  important  ways.  First,  it  is  a  survey  only  of  Navy  and 
Marine  Corps  ESSENCE  users.  Hence,  it  is  targeted  at  only  those  users  of  interest  to 
NMCPHC,  and  it  focuses  on  their  unique  Navy/Marine  Corps  perspectives  and  issues. 
Second,  the  survey  was  sent  to  all  existing  and  known  former  Navy  and  Marine  Corps 
users.  It  targeted  all  users,  not  just  those  who  have  the  resources  and  ability  to  attend 
professional  conferences  (such  as  International  Society  of  Disease  Surveillance  (ISDS)). 
Thus,  it  was  intended  to  garner  feedback  from  the  full  range  of  Navy  and  Marine  Corps 
ESSENCE  account  holders.  Third,  it  focused  on  areas  of  particular  interest  to  NMCPHC: 
how  the  Navy  and  Marine  Corps  monitors  used  the  ESSENCE  system,  what  their 
perceived  value  of  the  system  is,  and  what  training  they  would  like  to  see  in  the  future. 

The  survey  was  sent  to  225  Navy  and  Marine  Crops  users  with  either  an  active  or 
a  disabled  ESSENCE  account.  Ultimately,  143  of  the  users  responded  to  the  survey  for  a 
64  percent  response  rate.  Survey  findings  include  that,  overall,  nine  out  of  10  ESSENCE 
account  holders,  past  and  present,  favor  using  the  system,  find  it  valuable,  and  believe  the 
training  they  received  has  been  adequate.  However,  users  raised  four  issues:  1)  it  takes 


an  excessive  amount  of  time  to  obtain  an  account,  2)  passwords  are  required  to  be 
changed  too  often,  3)  there  are  too  many  miscodings,  leading  to  excessive  false  positive 
signals,  and  4)  training  and  training  tools  are  insufficient. 
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I. 


BIOSURVEILLANCE  AND  ESSENCE 


After  the  attacks  on  9/1 1,  the  United  States  (U.S.)  government  sought  out  ways  to 
protect  the  nation  from  future  terrorist  attacks.  One  form  of  a  potentially  very  deadly 
attack  is  bioterrorism.  These  attacks  can  be  in  the  form  of  biological  weapons,  perhaps 
consisting  of  an  infectious  disease  or  diseases  dispersed  throughout  the  country,  with  the 
goal  of  infecting  the  population  and  killing  members  of  our  society.  Examples  of 
bioterrorism  agents  include  anthrax,  botulism,  and  epsilon  toxin,  to  name  a  few.  Details 
of  these  agents  and  other  virulent  naturally  occurring  diseases  are  listed  on  the  Centers 
for  Disease  Control  and  Prevention’s  (CDC)  Website:  http://www.bt.cdc. gov/agent/ 
agentlist.asp.  To  help  protect  the  country  against,  and  mitigate  the  effects  of,  a 
bioterrorist  attack,  federal,  state,  and  local  agencies  conduct  biosurveillance. 

A.  WHAT  IS  BIOSURVEILLANCE? 

In  October  2007,  President  George  W.  Bush  issued  Homeland  Security 
Presidential  Directive  21  (HSPD-21,  2007).  It  defines  biosurveillance  as 

the  process  of  active  data-gathering  with  appropriate  analysis  and 
interpretation  of  biosphere  data  that  might  relate  to  disease  activity  and 
threats  to  human  or  animal  health — whether  infectious,  toxic,  metabolic, 
or  otherwise,  and  regardless  of  intentional  or  natural  origin — in  order  to 
achieve  early  warning  of  health  threats,  early  detection  of  health  events, 
and  overall  situational  awareness  of  disease  activity.  (HSPD-21,  2007) 

One  part  of  biosurveillance,  syndromic  surveillance,  “is  the  surveillance  of  health- 
related  data  that  precedes  diagnosis  to  detect  a  disease  outbreak  or  other  health  related 
event  that  warrants  a  public  response”  (Kleiner  et  al.,  2008).  Under  the  umbrella  of 
biosurveillance,  syndromic  surveillance  is  intended  to  provide  early  event  detection  of  an 
illness  or  attack  before  any  particular  individual  has  actually  been  diagnosed  with  the 
specific  illness  or  attack  agent.  It  is  also  intended  to  provide  situational  awareness  of  the 
location  and  spread  of  the  illness  or  agent  so  that  public  health  and  medical  professionals 
can  localize  and  treat  the  affected  population  or  community. 
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In  2003,  agencies,  such  as  the  CDC  and  state  and  local  health  departments,  were 
estimated  to  have  approximately  100  sites  throughout  the  country  that  utilize  a  syndromic 
surveillance  system  (Buehler  et  al.,  2003,  p.  1197). 

The  goal  of  these  systems  is  to  enable  earlier  detection  of  epidemics  and  a 
more  timely  public  health  response,  hours  or  days  before  disease  clusters 
are  recognized  clinically,  or  before  specific  diagnoses  are  made  and 
reported  to  public  health  authorities.  (Buehler  et  al.,  2003,  p.  1197) 

In  order  for  early  event  detection  to  occur,  a  syndromic  surveillance  or 
biosurveillance  systems  had  to  be  created.  On  the  other  hand,  a  year  later,  Dena  Bravata, 
MD  and  other  authors  synthesized  numerous  articles  and  concluded, 

Few  surveillance  systems  have  been  specifically  designed  for  collecting 
and  analyzing  data  for  the  early  detection  of  a  bioterrorist  event.  Because 
current  evaluations  of  surveillance  systems  for  detecting  bioterrorism  and 
emerging  infections  are  insufficient  to  characterize  the  timeliness  or 
sensitivity  and  specificity,  clinical  and  public  health  decision  making 
based  on  these  systems  may  be  compromised.  (Bravata  et  al.,  2004,  p. 

917) 

However,  in  2008,  Buehler  and  his  team  conducted  a  survey  of  public  health 
practitioners  in  which  over  two-thirds  of  the  survey  respondents  indicated  that  they  would 
expand  their  use  of  syndromic  surveillance  within  the  next  two  years  (Buehler,  2008). 

B.  THE  EARLY  NOTIFICATION  OF  COMMUNITY-BASED  EPIDEMICS 
(ESSENCE)  SYSTEM 

A  number  of  systems  have  been  produced  that  are  used  at  the  federal,  state  and 
local  levels.  These  include  the  Early  Aberration  Reporting  System  (EARS),  BioSense, 
The  National  Electronic  Disease  Surveillance  System  (NEDSS)  and  the  Electronic 
Surveillance  System  for  the  Early  Notification  of  Community-Based  Epidemics 
(ESSENCE).  Various  federal,  state  and  local  health  departments  operate  and  monitor 
these  systems;  however,  ESSENCE  is  the  only  system  used  by  the  Department  of 
Defense  (DoD). 
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EARS:  “Various  city,  county,  and  state  public  health  officials  in  the 
United  States  and  abroad  currently  use  EARS  on  syndromic  data  from 
emergency  departments,  reportable  conditions,  911  calls,  physician  office 
data,  school  and  business  absenteeism,  and  over-the-counter  drug  sales. 

EARS  is  convenient,  easy  to  use,  and  available  at  no  cost.”  (CDCb,  2010) 

BioSense:  “In  FY  2010,  CDC  started  redesigning  the  BioSense  program 
based  on  input  and  guidance  from  our  local,  state,  and  federal  partners. 

The  goal  of  the  redesign  effort  is  to  be  able  to  provide  nationwide  and 
regional  situational  awareness  for  all-hazard  health-related  threats  (beyond 
bioterrorism)  and  to  support  national,  state,  and  local  responses  to  those 
threats.  By  integrating  local  and  state-level  information,  CDC  will 
provide  a  timely  and  cohesive  picture  at  the  regional  (i.e.,  multistate)  and 
national  levels  and  improve  BioSense's  utility.”  (CDCa,  2010) 

NEDSS:  “is  an  initiative  that  promotes  the  use  of  data  and  information 
system  standards  to  advance  the  development  of  efficient,  integrated,  and 
interoperable  surveillance  systems  at  federal,  state  and  local  levels.  The 
vision  of  NEDSS  is  to  have  integrated  surveillance  systems  that  can 
transfer  appropriate  public  health,  laboratory,  and  clinical  data  efficiently 
and  securely  over  the  Internet.  NEDSS  will  revolutionize  public  health  by 
gathering  and  analyzing  information  quickly  and  accurately.  This  will  help 
to  improve  the  nation's  ability  to  identify  and  track  emerging  infectious 
diseases  and  potential  bioterrorism  attacks,  as  well  as  to  investigate 
outbreaks  and  monitor  disease  trends.”  (CDC) 

1.  The  History  of  ESSENCE 

On  June  12,  1996,  the  Clinton  Administration  released  a  fact  sheet  via  the  White 
House  Office  of  Science  and  Technology  (OST)  stating  that 

the  National  Science  and  Technology  Council  [had]  determined  that  the 
national  and  international  system  of  infectious  disease  surveillance, 
prevention,  and  response  is  inadequate  to  protect  the  health  of  U.S. 
citizens.  (OST  Policy,  1996) 

As  described  in  the  fact  sheet,  the  administration's  goals  were  to: 

1.  “Strengthen  the  domestic  infectious  disease  surveillance  and 
response  system,  both  at  the  Federal,  State,  and  local  levels  and  at 
ports  of  entry  into  the  United  States,  in  cooperation  with  the 
private  sector  and  with  public  health  and  medical  communities,” 

2.  “Establish  a  global  infectious  disease  surveillance  and  response 
system,  based  on  regional  hubs  and  linked  by  modem 
communications,”  and 
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3. 


“Expand  missions  and  establish  the  authority  of  relevant  United 
States  Government  agencies  to  contribute  to  a  worldwide 
infectious  disease  surveillance,  prevention,  and  response  network.” 

(OST  Policy,  1996) 

It  further  stated  that  the  United  States  Government  (USG)  roles  and 
responsibilities  were  to  “enhance  the  surveillance  and  response  components  of  our 
domestic  and  international  public  health  infrastructure  [by]  strengthening  the]  federal 
and  state  laboratory  and  epidemiological  response  capabilities.  The  CDC  will  coordinate 
federal  government  efforts  to  strengthen  federal,  state  and  local  health  departments’] 
surveillance  and  response  capabilities”  (OST  Policy,  1996). 

Following  OST's  direction,  the  Department  of  Defense  created  the  DoD  Global 
Emerging  Infections  Surveillance  and  Response  System  (DoD-GEIS)  in  1997.  With  a 
focus  on  global  health  surveillance,  the  DoD-GEIS  began  with  laboratories  in  countries, 
such  as  Peru,  Indonesia,  Egypt,  Thailand,  and  Kenya  (Sueker,  2010).  A  sponsored 
program  of  the  DoD-GEIS  oversees  the  Global  Influenza  Surveillance  Program  and 
ESSENCE  syndromic  surveillance  system.  “ESSENCE  had  been  [a]  developmental 
program  for  syndromic  surveillance  using  data  from  the  National  Capital  Area,  but 
following  the  events  of  September  11,  2001,  ESSENCE  began  receiving  data  from  all 
military  medical  facilities  in  DoD”  (Macintosh,  2004).  Thus,  the  first  implementation  of 
ESSENCE,  ESSENCE  I,  gathered  information  from  “military  personnel  and  their 
dependents  at  all  military  treatment  facilities  by  using  ambulatory  records  generated  for 
TriCare,  the  military's  health-care  system”  (Lombardo,  2004). 

The  next  variant,  “ESSENCE  II  was  a  regional  system  that  supported  advanced 
surveillance  within  the  National  Capital  Region  (NCR)  test  bed.  The  system  [was] 
developed  by  the  Johns  Hopkins  University  Applied  Physics  Laboratory  (JHU/APL)  in 
collaboration  with  the  Maryland  Department  of  Health  and  Mental  Hygiene,  the  District 
of  Columbia  Department  of  Health,  and  the  Virginia  Department  of  Health”  (Lombardo, 
2004).  The  DoD-GEIS  and  JHU/APL  also  collaborated  on  creating  and  updating 
subsequent  variants  of  the  ESSENCE  syndromic  surveillance  system.  Primarily,  their 
work  focused  on  the  algorithms  necessary  for  the  system  to  detect  syndromic  groupings 

before  an  epidemic  has  matured  into  a  health  catastrophe.  What  the  developers  of  the 
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system  accomplished  was  to  allow  “ESSENCE  [to]  capture  data  in  multiple  formats, 
parses  text  strings  into  syndrome  groupings,  and  applies  multiple  temporal  and  spatio- 
temporal  outbreak-detection  algorithms.  [In  a]  DARPA  evaluation  exercise,  ESSENCE 
algorithms  detected  a  set  of  health  events  with  a  median  delay  of  1  day  after  the  earliest 
possible  detection  opportunity”  (Lombardo,  2004). 

The  newest  installment  of  the  syndromic  surveillance  system,  ESSENCE  IV,  is 
currently  under  test  evaluations  and  is  expected  to  come  online  within  the  next  few  years. 

2.  Why  Does  the  DoD  and  the  U.S.  Navy  Use  ESSENCE? 

Although  civilian  health  departments  use  biosurveillance  systems,  the  DoD  has 
also  made  their  use  mandatory  as  well.  In  1997,  DoD  Directive  6490.2 — Joint  Medical 
Surveillance,  “establishes  policy  and  assigns  responsibility  for  routine  joint  medical 
surveillance  of  all  Military  Service  members  during  active  Federal  service,  especially 
military  deployments.”  Then,  in  2004,  DoD  Directive  62004 — Force  Health  Protection 
(FHP),  “assigns  responsibility  for  implementing  FHP  measures,  on  behalf  of  all  Military 
Service  members  during  active  and  Reserve  military  service,  encompassing  the  full 
spectrum  of  mission,  responsibilities,  and  actions  of  the  DoD  Components  in 
establishing,  sustaining,  restoring,  and  improving  the  health  of  their  forces.”  The  August 
24,  2009,  DoD  Directive  6490. 2E  incorporating  change  1,  originally  written  in  October 
2004,  states  that  “the  DoD  components  shall  conduct  comprehensive,  continuous,  and 
consistent  military  health  surveillance  to  implement  early  intervention  and  control 
strategies,  using  joint  technologies,  practices,  and  procedures  in  a  manner  consistent 
across  the  military  services.  Relevant,  timely,  actionable,  comprehensive  health 
surveillance  information  shall  be  collected  and  maintained  to  support  the  Armed  Forces.” 
Following  this  guidance  in  2009,  the  U.S.  Navy  and  the  Navy  Bureau  of  Medicine  and 
Surgery  (BUMED)  mandated  that  the  “monitoring  of  ESSENCE  alerts  applies  to  ah 
Navy  medical  treatment  facilities  (MTFs).  Health  surveillance  shall  be  conducted  to 
enable  early  intervention  and  control  strategies.  Monitoring  for  and  timely  reporting  of 
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significant  health  events  that  may  adversely  affect  mission  accomplishment  and  shape  the 
commander’s  decision  making  is  vital  military  medicine  functions”  (BUMEDINST 
6220. 12B  2009). 

3.  How  Does  the  ESSENCE  System  Work  and  Who  Uses  It? 

The  military  treatments  facilities  that  use  the  ESSENCE  system  capture  data  from 
the  military  health  system  (MHS),  which  includes  outpatient  clinical  visits,  pharmacy 
transactions,  and  laboratory  orders.  The  ESSENCE  system  then  organizes  the  ICD-9 
(International  Classification  of  Diseases  9th  Edition)  codes1  input  by  the  doctor,  nurse, 
laboratory  technician,  or  other  health  professionals,  and  analyzes  and  sorts  the  data  into 
syndromic  types  that  an  operator  of  the  system  can  view.  Via  the  use  of  statistical 
algorithms,  the  system  seeks  to  identify  emerging  or  re-emerging  infectious  diseases  that 
pose  a  substantial  risk  to  a  commander’s  mission  or  the  U.S.  Navy’s  mission.  The 
algorithms  are  intended  to  allow  the  user  to  see  increases  and  decreases  in  a  particular 
syndrome  so  they  can  monitor  its  activity  in  their  region. 

Again,  the  ESSENCE  system  is  used  for  Early  Event  Detection  (EED)  and 
location  situational  awareness  (SA).  The  CDC  definition  of  EED  is  “the  ability  to  detect, 
at  the  earliest  possible  time,  events  that  may  signal  a  public  health  emergency.  EED  is 
comprised  of  case  and  suspect  case  reporting  along  with  statistical  analysis  of  health- 
related  data.  Both  real-time  streaming  of  data  from  clinical  care  facilities,  as  well  as 
batched  data  with  a  short  time  delay  are  used  to  support  EED  efforts”  (CDC,  2008).  The 
definition  of  situational  awareness  is  “the  ability  to  utilize  detailed,  real-time  health  data 
to  confirm,  refute  and  to  provide  an  effective  response  to  the  existence  of  an  outbreak.  It 
also  is  used  to  monitor  an  outbreaks  magnitude,  geography,  rate  of  change  and  life  cycle” 
(CDC,  2008). 

Although  these  terms  are  similar  in  definition,  they  actually  provide  the  monitor 
with  two  mechanisms  towards  understanding  how  and  why  the  ESSENCE  program  was 

1  ICD-9  clinical  modification  codes  are  defined  as  “a  standardized  classification  of  disease,  injuries, 
and  causes  of  death,  by  etiology  and  anatomic  localization  and  codified  into  a  6-digit  number,  which  allows 
clinicians,  statisticians,  politicians,  health  planners  and  others  to  speak  a  common  language,  both  US  and 
internationally”  (The  Free  Dictionary,  2010). 
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developed  and  how  to  use  it  properly.  EED  allows  the  user  to  find  communicable 
diseases  that  are  on  the  rise  in  their  area  prior  to  a  doctor  diagnosing  the  event.  Because 
an  account  holder  is  able  to  monitor  multiple  military  treatment  facilities  at  one  time,  it  is 
possible  to  observe  increases  using  tools,  such  as  a  time-series  graph  (which  will  be 
explained  below)  and  alert  status  received  from  the  ESSENCE  system  itself.  Situational 
awareness,  on  the  other  hand,  gives  the  monitor  the  ability  to  observe  multiple  MTFs  and 
to  understand  which  already  diagnosed  illnesses  are  on  the  rise  and  continue  to  rise,  as 
well  as  be  able  to  monitor  the  patients  that  enter  the  MTFs  via  city,  state,  region,  and 
country. 

The  Navy  ESSENCE  monitors  are  health  professionals,  including  preventive 
medicine  technicians,  doctors,  nurses,  public  health  professionals,  and  epidemiologists. 
The  agencies  that  monitor  ESSENCE  are  the  preventive  medicine  and  public  health 
departments  of  the  Navy  throughout  the  world.  They  are  comprised  of  naval  clinics, 
naval  hospitals,  and  naval  medical  centers.  The  Navy  Environment  Preventive  Medicine 
Units  (NEPMU),  which  are  located  in  San  Diego,  California,  Pearl  Harbor,  Hawaii,  and 
Norfolk,  Virginia,  also  monitor  disease  activity  using  ESSENCE.  Finally,  the  Navy  and 
Marine  Corps  Public  Health  Center  (NMCPHC)  oversees  patient  syndromic  surveillance 
throughout  the  Navy  as  mandated  by  the  BUMEDFNST  6220. 12B. 

C.  WHY  SURVEY  ESSENCE  ACCOUNT  HOLDERS? 

In  accordance  with  their  BUMED  mandate,  the  NMCPHC  wants  to  better 
understand  how  ESSENCE  is  being  used  by  surveying  Navy  and  Marine  Corps 
ESSENCE  account  holders  about  their  training,  employment,  and  perceived  value  of  the 
system.  For  example,  NMCPHC  has  noticed  that  some  installations  that  are  supposed  to 
be  viewing  and  using  the  ESSENCE  system  on  a  daily  basis  are  not.  Thus,  they  want  to 
understand  the  reason  why  this  phenomenon  is  occurring.  In  addition,  the  NMCPHC  has 
little  information  about  whether  their  users  understand  how  to  use  the  system  effectively 
and  efficiently  or  whether  users  think  the  system  is  providing  them  with  the  proper 
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information  they  need.  Furthermore,  the  NMCPHC  is  interested  in  monitors'  training  and 
usage  habits  of  biosurveillance  and/or  syndromic  surveillance. 

1.  Related  Research 

In  2007-2008,  the  International  Society  for  Disease  Surveillance  (ISDS)  and  the 
CDC  sponsored  a  survey  of  59  state,  territorial,  and  selected  large  local  jurisdictions  in 
the  United  States  regarding  their  use  of  syndromic  surveillance.  The  survey  defined 
syndromic  surveillance  as  systems  with  all  of  the  following  characteristics: 

•  surveillance  for  human  health-related  events  or  outcomes; 

•  surveillance  for  the  purpose  of  early  event  detection  or  situational 
awareness; 

•  ongoing  surveillance  as  opposed  to  time-limited,  ‘drop-in’  surveillance 
around  specific  high-profile  events;  and, 

•  surveillance  systems  not  established  primarily  to  support  notifiable  disease 
reporting  (Buehler  et  ah,  2008). 

The  ISDS  survey  was  e-mailed  to  respondents  and  telephone  notifications  made. 
A  remarkably  high  response  rate  of  88  percent  was  achieved,  and  the  responding 
jurisdictions  accounted  for  94  percent  of  the  U.S.  population.  Eighty-three  percent  of 
respondents  “reported  conducting  syndromic  surveillance  for  a  median  of  3.3  years. 
Emergency  data  visits  were  [a]  commonly  reported  information  source  (84  percent). 
Seventy-two  percent  reported  reviewing  the  data  from  participating  hospitals  at  least 
daily”  (Buehler,  2008).  Ninety-three  percent  reported  that  syndromic  surveillance  was 
“highly  useful  or  somewhat  useful”  for  “monitoring  influenza”  (Buehler,  2008). 

Moreover,  syndromic  surveillance  was  concluded  to  be  more  useful  when 
tracking  and  ILI  and  other  larger  communicable  diseases  rather  than  smaller  or  rarer 
diseases.  Over  two-thirds  of  the  respondents  indicated  that  they  would  expand  their  use 
of  syndromic  surveillance  within  the  next  two  years  (Buehler,  2008).  Buehler  and  his 
team  of  researchers  deduced  that  syndromic  surveillance  would  continue  to  expand  and 
that  it  is  important  to  provide  future  users  training  and  lessons  learned  on  how  to  monitor 
their  systems  correctly. 
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The  Johns  Hopkins  Applied  Physics  Laboratory  (JHU/APL)  conducted  another 
survey  in  2009  on  behalf  of  the  Armed  Forces  Health  Surveillance  Center  (AFHSC). 
Their  objective  was  to  “examine  the  strengths  and  weaknesses  of  the  current  ESSENCE 
system,  by  capturing  feedback  from  current  DoD  ESSENCE  users”  (JHU/APL,  2009). 
JHU/APL  developed  a  survey,  which  they  entitled  the  “DoD  ESSENCE  User  Feedback 
Tool”  and  the  AFHSC  sponsored  a  workshop  during  the  2009  Force  Health  Protection 
Conference.  Thirty-one  DoD  ESSENCE  users  participated  in  the  workshop  and  17  DoD 
ESSENCE  users  responded  to  their  survey.  Although  the  number  of  monitors  was  low, 
they  were  able  to  obtain  worthy  and  significant  feedback  because  those  who  did 
participate  voiced  their  concerns  honestly  and  openly  (JHU/APL,  2009).  JHU/APL 
concluded  that  the  most  important  issue  for  ESSENCE  is  a  “need  for  user  training  both  in 
how  and  when  to  use  the  functions  available  in  the  tool,  as  well  as  how  to  interpret  the 
resulting  system  displays.”  They  further  concluded  that  the  “second  most  raised  issue, 
with  regard  to  the  currently  deployed  system,  was  the  impact  that  incorrectly  coded 
information,  as  well  as  delays  in  the  coding  information  has  on  both  the  timeless  and 
usefulness  of  the  information  displayed  and  the  resulting  alters  generated  by  the  system” 
(JHU/APL,  2009). 

2.  How  Is  This  Survey  Different? 

This  survey,  conducted  on  behalf  of  the  NMCPHC,  differs  from  the  JHU/APL 
survey  in  a  number  of  important  ways.  First,  only  Navy  and  Marine  Corps  ESSENCE 
users  were  surveyed.  Hence,  it  targeted  only  those  users  of  interest  to  NMCPHC  and 
focused  on  their  unique  Navy/Marine  Corps  perspectives  and  issues.  Second,  the  survey 
was  sent  to  all  existing  and  known  former  Navy  and  Marine  Corps  users,  225  in  all. 
Therefore,  it  targeted  all  users,  and  not  just  those  with  the  resources  and  ability  to  attend 
professional  conferences,  such  as  ISDS.  Thus,  it  was  intended  to  gamer  feedback  from 
the  full  range  of  Navy  and  Marine  Corps  ESSENCE  users.  Third,  it  focused  on  areas  of 
particular  interest  to  NMCPHC:  how  the  Navy  and  Marine  Corps  monitors  used  the 
ESSENCE  system,  what  their  perceived  value  of  the  system  was,  and  what  training  they 
would  like  to  see  in  the  future. 
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D.  ORGANIZATION  OF  THIS  THESIS 


This  thesis  in  organized  as  follows.  Chapter  II  describes  the  background  research 
and  how  the  survey  instrument  was  designed.  Chapter  III  describes  the  survey  approval 
process,  how  the  survey  was  pretested  and  fielded,  and  the  survey  response  rate.  Chapter 
IV  presents  the  survey  results.  Finally,  Chapter  V  summarizes  the  findings  and  provides 
recommendations  for  steps  the  NMCPHC  can  take  towards  improving  the  ESSENCE 
system. 
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II.  SURVEY  DESIGN 


This  chapter  discusses  my  background  research  and  how  the  survey  instrument 
was  designed.  See  Appendix  A  for  a  copy  of  the  survey  instrument. 

A.  BACKGROUND  RESEARCH 

To  understand  fully  how  monitors  utilize  the  system,  I  traveled  to  several  MTFs 
around  the  country.  In  Portsmouth,  Virginia,  I  spoke  to  the  (former)  ESSENCE  Program 
Officer,  who  demonstrated  the  functionalities  of  the  program,  such  as  the  alert  list,  the 
reportable  and  syndromic  queries,  site  selections,  and  the  alert  e-mails.  Moreover, 
instruction  was  given  on  how  a  monitor  should  use  the  system  effectively  to  define 
syndromes  to  detect  a  particular  disease  early  in  its  maturity  and  also  for  the  situational 
awareness  of  a  disease  as  well.  She  also  identified  how  miscoded  ICD-9’s  force  the 
monitor  to  investigate  outside  sources,  such  as  the  Composite  Health  Care  System 
(CHCS)  and  Armed  Forces  Health  Longitudinal  Technology  Application  (AHLTA),  to 
discern  the  proper  identifying  code  for  the  patient  and  assess  if  it  contributes  to  an 
increase  in  a  communicable  disease  in  a  specific  area. 

Following  the  visit  to  Virginia,  I  created  a  questionnaire  to  ask  the  monitors 
particular  questions  during  future  visits.  The  questionnaire  was  comprised  of  desired 
responses  from  my  sponsor  and  questions  I  knew  would  help  me  understand  what  a 
monitor  does  on  a  daily  basis  and  what  can  be  modified  for  future  ESSENCE  use. 

I  then  met  with  monitors  at  NEPMU-5,  the  Naval  Hospital  (NH)  Camp  Pendleton, 
and  the  Naval  Medical  Center  (NMC)  San  Diego.  First,  I  began  by  speaking  with  the 
regional  monitor  at  NEPMU-5  and  learned  that  regional  monitors  use  a  wider  site 
selection  criterion  to  track  the  communicable  diseases  on  the  rise  in  their  region.  The 
regional  monitor  stated  that  the  role  of  a  regional  monitor  is  different  from  local 
monitors.  She  does  not  investigate  alerts;  however,  she  does  oversee  that  her  region’s 
monitors  are  accurately  investigating  alerts  and  reporting  outbreaks  up  their  chain  of 
command. 
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Next,  I  visited  local  monitors  from  the  NH  Camp  Pendleton  and  from  the  NMC 
San  Diego.  My  last  visit  was  to  Jacksonville,  Florida,  where  I  met  with  my  sponsor,  Mrs. 
Asha  Riegodedios  at  the  NMCPHC.  Before  the  meeting,  I  spoke  on  the  phone  with  two 
local  monitors  from  Naval  Air  Station  Pensacola,  Florida.  The  local  MTF  users  provided 
similar  types  of  feedback.  They  also  solidified  my  understanding  of  how  a  local  sponsor 
works  ESSENCE  into  their  daily  routine  and  what  drives  them  to  investigate  alerts  and 
the  methods  used  to  conduct  the  investigations.  An  important  note  from  these  monitors  is 
that  they  were  only  taught  a  small  amount  of  what  biosurveillance  is  in  C-school,  and 
thus,  had  to  learn  the  capabilities  and  functionalities  of  the  system  via  hands-on  training. 
The  one  of  the  most  crucial  take-aways  I  received  from  these  users  was  that  they  spend  a 
lot  of  time  investigating  alerts  in  ESSENCE  by  using  CHCS  and  AHLTA.  They  are 
required  to  confirm  that  an  alert  is  real  and  if  the  trend  in  that  particular  communicable 
disease  is  on  the  rise. 

However,  when  asked  if  the  various  users  had  ever  identified  an  outbreak  by 
monitoring  ESSENCE  alerts,  half  said  they  did  and  half  said  they  did  not.  Those  who  did 
identify  outbreaks  said  that  this  was  the  very  reason  they  valued  the  program.  By  digging 
through  the  data  details  to  specify  site  selections,  date  intervals,  and  syndromes,  and 
coordinating  their  efforts  tracking  patient  history  in  ALTHA,  they  could  track  where  the 
outbreak  originated.  The  value  the  local  MTF  monitors  have  for  ESSENCE  also  varied. 
A  few  indicated  that  it  provides  excellent  situational  awareness  as  a  means  of  detecting 
rises  in  communicable  diseases.  ESSENCE  provides  them  an  “X”  that  marks  the  spot 
and  through  ALTHA  and  CHCS,  they  are  able  to  dig  down  through  the  data  to  discover  if 
an  actual  outbreak  is  occurring  in  their  community. 

On  the  other  hand,  some  believed  the  system  has  tremendous  value,  but  at  times 
alerts  appear  when  they  should  not,  due  to  either  miscodings  or  routine  work  done  by 
doctors  in  a  particular  hospital  specializing  in  a  certain  disease.  Also,  the  system 
considers  the  days  that  the  hospital  or  clinic  is  closed  as  zero  visits  that  day,  which  is 
inaccurate  and  drives  the  running  average  down  over  the  “14-day  period,  making  it 
problematic  as  it  should  be  longer  for  a  more  accurate  baseline.” 
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After  speaking  with  and  discussing  the  system  with  actual  monitors,  I  concluded 
that  there  are  varied  opinions  about  the  utility  and  value  of  the  ESSENCE  program,  and 
therefore,  these  aspects  must  be  looked  addressed  in  the  survey.  In  particular,  how  many 
ESSENCE  monitors  have  these  same  opinions  and  how  many  monitors  hold  differing 
opinions?  This  then  became  the  heart  of  the  research.  My  meeting  with  Mrs. 
Riegodedios  consisted  of  discussing  these  particular  comments  that  the  monitors  said 
about  their  use  and  value  of  the  system,  and  what  changes  they  would  like  to  make.  It 
was  then  possible  to  compile  a  few  notes  and  specific  topics  that  the  survey  needed  to  be 
addressed  by  the  survey. 

B.  SURVEY  INSTRUMENT  DESIGN 

1.  Specifying  the  Survey  Objective 

In  her  paper,  “Fundamental  of  Survey  Research  Methodology,  Priscilla  Glasow 
writes,  “survey  instrument  development  must  be  preceded  by  certain  prerequisites.  First, 
the  focus  of  the  study  must  be  carefully  defined.  Second,  the  study  objectives  must  be 
translated  into  measurable  factors  that  contribute  to  that  focus”  (Glasow,  2005,  pp.  2-5). 
It  follows  that  writing  the  objective  of  ones’  survey  first  helps  to  outline  the  layout  and 
flow  of  the  survey. 

Therefore,  after  I  had  completed  the  research  travel,  I  had  some  idea  of  what 
ESSENCE  monitors’  day-to-day  experiences  were,  of  what  their  training  might  or  might 
not  be  comprised  of,  and  what  they  believed  the  ESSENCE  system  provided  them. 
Given  this  information,  and  considering  the  information  my  sponsor  wanted  to  learn  from 
the  survey,  as  well  as  what  I  thought  the  survey  should  include,  I  specified  three  main 
survey  foci:  usage,  value,  and  training.  Thus,  the  survey  objective  became  the  following: 
to  assess  ESSENCE  account  holder’s  usage  of,  perceived  value  of,  and  training  of  the 
system. 


2.  Designing  the  Survey  Instrument 

Given  the  objective,  I  then  began  drafting  the  survey  questions.  Initially  many  of 
the  questions  were  overly  wordy,  sometimes  less  than  completely  coherent,  and  the 
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question  structures  and  formats  were  mixed  around.  For  instance,  the  questions  that 
pertained  to  value  were  preceded  by  the  following:  Do  you  agree  or  disagree  with  the 
following  statements  regarding  the  ESSENCE  program?  This  question  was  subsequently 
modified  because  it  only  allowed  a  binary  response  and  thereby  limited  the  amount  of 
information  a  respondent  could  convey  about  the  system's  value.  In  the  final  version  of 
the  survey,  the  question  read:  How  much  do  you  agree  or  disagree  with  the  following 
statements  regarding  the  ESSENCE  program?  The  subtle  difference  is  in  the  format  of 
the  question,  where  the  final  survey  allows  the  respondent  to  answer  naturally  on  a  5- 
point  Likert  scale. 

Given  a  set  of  draft  questions,  Professor  Don  Dillman,  a  leading  expert  on  survey 
methodology,  says  that  effort  should  be  “made  to  order  questions  in  a  way  that  will  be 
logical  to  the  respondent”  (Dillman  et  al.  2008,  p.  88).  In  the  first  design  of  the  survey, 
the  question  structure  was  found  to  be  confusing  to  respondents  and  did  not  flow 
properly.  Also,  the  format  of  the  questions  that  would  allow  me  to  assess  monitor  usage 
was  disheveled  and  lacked  a  chronological  ordering.  For  instance,  questions  about  time- 
series  graphs  and  queries  were  hidden  within  the  middle  of  the  survey,  rather  than 
arranged  in  the  order  that  respondents  found  most  natural:  alert  e-mails,  queries, 
investigations,  and  then  data  details.  The  use  of  time-series  graphs,  in  my  opinion,  is 
better  asked  under  the  category  of  perceived  value  because  it  has  the  potential  to  assist 
the  monitor  in  identifying  outbreaks,  and  thus,  providing  situational  awareness  advertised 
by  the  proponents  of  syndromic  surveillance.  However,  shifting  the  questions  around  so 
they  occur  in  the  natural  ordering  as  if  someone  where  to  log  into  ESSENCE  and  conduct 
their  daily  alert  and  query  check,  helped  guide  the  respondent  in  remembering  how  they 
use  ESSENCE. 

Finally,  I  gave  some  thought  about  how  to  start  the  survey.  As  Professor  Dillman 
states,  “choose  the  first  question  carefully”  (Dillman  et  al.,  2008,  p.  92).  He  continues  by 
saying  that  it  should  be  “easy,”  “interesting,”  and  “apply  to  everyone”  (Dillman  et  al., 
2008,  p.  92).  The  first  question  does  just  that:  “On  average,  how  often  do  you  log  into 
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ESSENCE?”  From  this  first  question,  the  survey  then  flowed  through  five  major 
categories  of  questions:  monitor  usage,  perceived  value,  user  training,  use  of  the  system, 
and  demographics.  The  following  subsections  describe  each  of  these  question  categories. 

a.  Assessing  Monitor  Usage 

The  background  research  I  gained  through  traveling  and  speaking  with 
different  monitors  throughout  the  country  provided  insight  as  to  how  this  section  of  the 
survey  must  be  developed.  Understanding  their  log-in  habits  and  module  usage 
contributed  towards  assessing  monitor  usage.  For  instance,  the  survey  begins  with  the 
following  question:  “On  average,  how  often  do  you  log  into  ESSENCE?”  This  first 
question  draws  the  participant  into  the  survey,  it  is  easy  to  answer,  and  it  begins  with 
logging  into  the  program.  The  next  set  of  questions  asks  the  monitors  whether  they 
receive  e-mail  alerts  from  ESSENCE  and  if  so,  if  that  is  the  reason  they  are  logging  in. 
The  next  set  of  questions  ask  monitors  if  they  use  the  alert  list,  reportable  diseases,  and 
syndromic  query  modules  and  what  they  functionalities  they  use  and  why  they  perform 
them  while  conducting  surveillance  and  SA  in  those  modules.  In  the  last  part  of  this 
section,  questions  were  asked  to  gauge  which  users  investigate  alerts  and  if  so,  how  many 
use  ESSENCE  to  investigate  those  alerts  and  what  site  selections,  data  details,  external 
tools,  and  recording  methods  do  they  use.  Next,  in  this  section,  the  survey  strives  to  elicit 
how  monitors  use  the  different  modules  to  view  disease  activity.  The  three  different 
modules  are  the  alert  list,  the  reportable  diseases  query,  and  the  syndromic  query. 

The  goal  of  this  section  is  to  help  the  NMCPHC  understand  the  usage 
patterns  of  the  monitor  and  ascertain  how  they  utilize  the  different  functionalities  of  the 
system  during  their  logged  in  time.  Since  the  alert  identifies  what  syndromic  diseases 
have  increased  in  a  particular  area,  or  site  selection,  then  it  can  be  understood  if  the  user 
wants  to  see  the  general  disease  count  in  that  area.  The  alert  list  is  perfect  for  EED 
queries  within  a  health  department.  A  syndromic  query  allows  the  user  to  view  specific 
diseases  within  a  certain  date  range  and  location.  By  specifying  a  particular  syndrome, 
for  instance  the  influenza-like-illness  (ILI)  syndrome,  the  monitor  can  visually  look  at  the 
syndrome’s  increased  and  decreased  daily  count  over  time  for  situational  awareness 
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purposes.  If  the  commander  of  a  Naval  Medical  Center  wanted  to  know  when  to  begin 
administering  flu  shots,  the  preventive  medicine  technician  would  need  to  download  the 
time-series  table  for  the  syndrome  for  their  hospital  and  look  at  the  when  the  flu 
increased  in  outpatient  care  to  give  with  certainty  when  the  next  epidemic  of  the  flu 
might  occur  this  year.  Using  past  history  is  a  useful  way  for  any  MTF  to  gain  SA  within 
their  area.  Lastly,  the  reportable  diseases  query  allows  the  Navy  monitor  to  collaborate 
with  their  fellow  Navy  Disease  Reporting  System  Internet  (NDRSi)2  account  holder. 
This  allows  the  two  individuals  to  confirm  that  the  24-hour  reportable  diseases,  as  well  as 
the  diseases  that  must  be  reported  within  30  days,  have  been  sent  up  their  chain  of 
command  via  the  NDRSi  system. 

Finally,  comprehending  who  investigates  alerts  provided  the  data  essential 
to  understanding  which  monitors’  spend  time  navigating  through  the  program  and 
assessing  true  outbreak  detection  or  false  positives.  The  question  that  needs  to  be  asked 
here  is  if  ESSENCE  is  used  to  provide  their  chain  of  command  with  outbreak  information 
or  EED  information  and  SA. 

It  is  possible  to  correlate  many  of  these  questions  with  the  demographics 
of  the  participants  to  understand  which  rank  and  professional  background/position  access 
a  particular  module.  For  instance:  do  preventive  medicine  technicians,  who  are  on 
average  mid-grade  enlisted  personnel,  regularly  use  the  program  compared  to,  say,  a 
high-ranking  officer,  such  as  the  commander  of  the  preventive  medicine?  Furthermore, 
answers  to  questions,  such  as  are  users  with  advanced  training  through  the  use  of 
ESSENCE  rather  than  those  with  graduate  degrees  understand  the  nuances  of  the  system 
better  and  comprehend  the  backbone  of  the  system  better?  It  is  also  important  to  discover 
what  the  daily  users  would  like  to  see  changed  within  the  program  so  that  their  time  spent 
logged  into  ESSENCE  is  efficient  and  effective. 


2  As  per  BUMED  INST  6220. 12B,  Medical  Event  Reporting  (MERs)  are  “required  for  select  diseases 
and  injuries  due  to  their  potential  to  compromise  operational  readiness,  present  hazards  to  the  military  or 
civilian  community,  result  in  quarantine,  or  generate  inquires  to  the  Chief,  BUMED.”  The  preferred 
method  for  MER  submittal  is  by  using  the  Web-based  NDRSi  system. 
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b.  Assessing  Perceived  Value 

Respondents'  perceived  value  of  the  ESSENCE  system  is  difficult  to 
assess  via  questions  in  a  survey.  If  the  surveyor  asks  the  respondents  outright  if  they 
value  the  system,  the  response  may  be  different  than  asking  them  an  array  of  questions 
that  represent  what  the  system  should  be  doing  for  the  user  and  if  it  is  providing  them 
with  the  results  for  which  it  was  originally  intended.  As  mentioned  before,  ESSENCE 
account  holders’  opinions  are  varied,  and  their  value  in  the  system  to  identify  outbreaks 
correctly  and  provide  situational  awareness  is  important  to  the  NMCPHC. 

The  first  iteration  of  the  survey  asked  whether  the  account  holders  value 
the  system  and  the  format  of  this  section  was  not  appropriately  positioned.  After  the  final 
iteration  of  the  survey,  the  questions  were  reformatted  to  explore  the  respondents’ 
understating  of  outbreak  detection  while  using  ESSENCE,  how  they  handle  miscodings, 
their  opinions  of  the  system  itself,  which  were  derived  from  the  NMCPHC ’s  Website, 
which  displays  the  strategy  and  policy  of  the  ESSENCE  system:  http://www- 
nehc.med.navy.mil/Preventive  Medicine/Disease  Surveillance/essence.aspx.  The  user 
guidance  can  be  found  on  this  page  as  well  and  it  also  helped  to  formulate  questions  that 
made  it  possible  to  understand  the  opinions  of  the  monitors.  Finally,  how  their 
experiences  in  obtaining  their  current  ESSECE  are  considered. 

Thus,  in  the  final  survey,  a  four-question  set  was  written  to  understand 
user  value.  The  first  asked  the  respondents  if  they  had  discovered  an  actual  outbreak 
using  ESSENCE  and  the  next  question  asked  which  outbreak  types  they  discovered  while 
using  ESSENCE.  The  next  question  set  asked  if  monitors  observed  miscoded  ICD-9 
codes  and  what  they  did  to  fix  them  in  the  future.  This  was  followed  by  a  question  that 
asked  monitors  if  they  agreed  or  disagreed  with  statements  regarding  the  ESSENCE 
program.  Finally,  the  last  set  of  questions  asked  how  long  it  took  the  monitors  to  acquire 
their  current  ESSENCE  account,  if  they  had  to  apply  more  than  once,  and  if  they  had  to 
speak  with  an  ESSENCE  helpdesk  representative  while  waiting  to  access  their  current 
account. 
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The  most  relevant  question  set  in  this  entire  survey  was  to  ascertain  which 
portion  of  the  population  discovered  an  actual  outbreak  using  ESSENCE.  The  next 
question,  on  the  same  page,  asked  the  respondent  to  check  all  the  outbreak  types 
discovered  using  ESSENCE.  After  much  debate,  this  question  was  not  skipped  if  the 
respondents  answered  no  to  the  first  question,  but  allowed  the  individual  to  answer  if  they 
choose  to  do  so.  The  interesting  question  that  developed  was  whether  the  individual 
would  answer  no  to  detecting  an  actual  outbreak  and  checking  the  types  of  outbreaks  they 
discovered  using  the  system.  If  they  answered  no,  but  checked  outbreaks,  did  they  find 
ghost  outbreaks  or  false  positives  that  were  not  actually  there?  Did  they  not  fully 
understand  whether  the  outbreak  occurred  or  not?  If  account  holders  were  discovering 
false  outbreaks,  then  perhaps  they  did  not  fully  understand  how  the  system  worked  or 
was  system  too  sensitive  for  users. 

Miscodings  were  put  into  the  perceived  value  category  because  if  users 
were  spending  too  much  time  deciphering  miscodings,  then  they  were  not  likely  to  value 
the  system  because  they  were  wasting  their  time  eliminating  the  miscodings  from  their 
collected  information  and  must  deduct  the  information  in  their  head,  rather  than  having 
ESSENCE  remove  the  miscodings  itself.  This  could  also  lead  to  false  outbreak 
detections  that  lower  the  value  of  the  system  and  lower  the  SA  and  EED  of  the  system  as 
well.  Perhaps  comments  pertaining  to  how  respondents  would  like  to  correct  these 
mistakes  might  be  viable  to  the  longevity  of  the  system  by  its  users  and  the  DoD  in 
general. 

The  monitors’  opinions  of  the  ESSENCE  system  are  vital  to  understanding 
how  they  perceive  its  value.  Thus,  the  respondents  were  asked  if  they  agreed  or 
disagreed  with  a  set  of  eight  questions  with  5-point  Likert  response  scale.  If  they  used 
the  system  how  the  NMCPHC  perceived  them  to  use  it,  then  their  value  in  the  system 
was  higher  than  those  who  disagreed  with  the  set  of  questions.  These  questions  were 
reformatted  to  explore  the  respondents’  opinions  of  the  system,  which  were  derived  from 
the  NMCPHC’s  Website,  which  displays  the  strategy  and  policy  of  the  ESSENCE  system 
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http://www-nehc.med.navv.mil/Preventive  Medicine/Disease  Surveillance/essence. aspx. 

The  user  guidance  can  be  found  on  this  page  as  well  and  it  also  helped  to  formulate 
questions  that  allowed  me  to  understand  the  opinions  of  the  monitors. 

Due  to  the  transient  nature  of  the  military,  if  monitors  go  overseas  in 
support  of  the  Navy’s  public  health  and  preventive  medicine  missions,  and  if  they 
experience  hardships  to  reacquire  their  accounts,  then  their  value  in  the  system  could  be 
diminished  and  they  might  not  want  to  waste  their  busy  day  trying  to  access  the  system. 
The  survey  assisted  in  correlating  whether  the  ease  of  regaining  an  account  after  it  had 
been  deactivated  would  increase  the  monitor’s  value  of  the  system  and  their  willingness 
to  use  it. 

In  my  opinion,  this  analysis,  we  expect  to  correlate  which  rank  and 
professional  background/position  observed  an  actual  outbreak  using  ESSENCE  and  who 
has  seen  mostly  false-positives.  If  miscodings  have  been  corrected,  then  monitors  should 
see  more  value  in  the  system  in  the  future.  Furthermore,  if  the  system  were  being  used  as 
originally  intended,  then  ESSENCE  would  be  valued.  However,  for  those  who  disagree 
with  these  statements,  who  are  these  monitors  and  what  comments  have  they  made  to 
make  ESSENCE  more  effective  and  efficient  to  use? 

c.  Assessing  User  Training 

In  this  last  major  section  of  the  survey,  questions  were  devised  to  assess 
user  training  to  understand  what  has  been  done  in  the  past  and  what  can  be  improved 
upon  for  future  users.  The  first  question  asked  the  monitors  where  they  received  their 
ESSENCE  training.  The  next  set  of  questions  ask  where,  how,  and  what  the  user  would 
like  to  see  provided  for  training  in  support  of  current  and  future  users  of  the  ESSENCE 
system. 

The  Navy  and  Marine  Corps  Public  Health  Center  wants  to  provide 
ESSENCE  users  with  the  best  training  possible.  First,  the  questions  were  formatted  to 
understand  what  their  past  training  has  been,  then  to  ascertain  what  they  would  like  to  see 
in  the  future.  Knowing  where  respondents  received  their  training  from  impacts  their 
value  of  the  system  and  their  ability  to  use  the  system  effectively  and  efficiently.  If  the 
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NMCPHC  can  provide  the  multiple  resources  to  teach  future  users,  the  questions  within 
the  survey  can  help  direct  them  towards  that  goal.  Three  questions  achieve  the  what, 
where,  why  and  how  aspects  for  future  learning  tools,  modes,  and  methods. 

I  expected  to  learn  that  most  users  had  been  self-taught  on  the  system  and 
they  would  like  to  attend  conferences  and  office  meetings  to  learn  about  ESSENCE  and 
its  functionalities  in  particular  cases,  such  as  outbreak  scenarios  and  how  to  evaluate 
alerts  properly. 


d.  Ease  and  Effective  Use  of  the  ESSENCE  System 

This  section  was  not  a  major  category;  however,  these  open-ended 
comment  fields  allowed  the  respondent  to  freely  explain  what  they  would  change  if  they 
could  and  to  provide  their  opinions  about  what  they  believed  made  the  system  easy  to 
use.  Also,  a  final  open-ended  comment  field  allowed  the  monitors  to  provide  any  other 
information  about  the  ESSENCE  system,  particularly  topics  and  issues  not  asked  about  in 
the  survey,  that  they  felt  were  important. 

e.  Demographics 

It  is  common  to  place  demographic  questions  at  the  end  of  a  survey 
because  these  questions  are  likely  to  be  the  least  important.  The  goal  is  to  minimize  the 
amount  and  type  of  information  lost  should  a  respondent  fail  to  complete  the  survey. 
Although  not  important,  as  stated  previously,  correlations  can  be  made  throughout  the 
major  categories  and  demographics  to  tie  the  results  together  and  make  them  clearer 
during  the  analysis  phase.  For  example,  knowing  how  many  MTFs  the  user  monitors 
allows  the  NMCPHC  to  understand  how  much  time  can  be  spent  on  ESSENCE  by  the 
monitor  and  how  much  training  they  need  in  the  future.  Furthermore,  their  professional 
backgrounds  provide  insights  into  the  training  the  user  may  have  had  about  syndromic 
surveillance  and  what  it  means  for  DoD,  the  Navy,  and  the  command. 
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III.  FIELDING  THE  SURVEY 


This  chapter  describes  the  required  survey  approvals,  how  the  survey  was 
pretested  and  fielded,  and  the  survey  response  rate. 

A.  OBTAINING  THE  REQUIRED  APPROVALS 

Prior  to  conducting  any  survey,  approval  must  be  obtained.  For  surveys  within 
the  confines  of  NPS,  at  a  minimum,  approval  must  be  obtained  from  the  NPS  Institutional 
Review  Board  (IRB).  In  addition,  since  this  survey  concerns  Navy  and  Marine  Corps 
personnel  outside  of  NPS,  approval  must  be  obtained  by  the  appropriate  survey  authority 
within  the  Navy. 

1.  NPS  Institutional  Review  Board  Approval 

Before  the  survey  could  be  administered  to  the  Navy  and  Marine  Corps  monitors, 
approval  had  to  be  obtained  from  the  NPS  IRB.  IRB  approval  is  required  for  any 
research  effort  involving  intervention  or  interaction  with  individuals  part  of  a  systematic 
investigation  and  designed  to  contribute  to  generalized  knowledge.  All  of  these 
conditions  applied  to  this  research  where,  by  definition,  a  survey  is  part  of  a  systematic 
investigation  that  requires  interaction  with  individuals,  and  this  research  is  specifically 
intended  to  result  in  generalized  knowledge  about  the  ESSENCE  system  in  particular  and 
biosurveillance  in  general. 

The  purpose  of  the  IRB  is  to  protect  individuals  involved  in  any  research  to 
ensure  that  the  research  is  conducted  in  an  ethical  manner,  as  well  as  that  all  individuals 
participating  in  the  research  are  fully  and  correctly  informed  about  the  research  and 
consent  to  participate  in  said  research.  Per  SECNAVINST  300. 39D,  “the  rights,  welfare, 
interests,  privacy,  confidentiality,  and  safety  of  human  subjects  shall  be  held  paramount 
at  all  times  ands  all  research  projects  shall  be  conducted  in  a  manner  that  avoids  all 
unnecessary  physical  or  mental  discomfort,  and  economic,  social  or  cultural  harm.” 
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Furthermore,  NAVPGSCOLINST  300.4  outlines  the  necessity  to  protect  volunteers  from 
undue  burden  during  an  experiment  or  research.  Also,  the  instruction  outlines  the 
mandatory  documents  that  must  be  included  in  a  packet  sent  to  the  IRB  chair. 

The  IRB  application  requested  approval  to  execute  the  survey.  The  application 
describes  the  study  design,  including  how  the  data  was  collected  and  safeguarded,  the 
risks  associated  with  the  research,  the  steps  taken  to  minimize  risks,  and  to  protect  the 
subject’s  welfare  and  the  potential  benefits  of  this  research.  NPS  IRB  approval  was 
obtained  on  October  22,  2010.  See  Appendix  B  for  the  IRB  approval  letter,  NMCPHC 
approval  letter,  and  other  IRB  documents. 

2.  OPNAV  N-l  and  BUMED  Approval 

Since  this  survey  targeted  personnel  outside  of  NPS,  Navy  survey  approval  was 
also  required.  OPNAV  Instruction  5300. 8C — Coordination  and  Control  of  Personnel 
Surveys  defines  the  approval  process  and  it  “applies  to  all  surveys  of  the  Department  of 
the  Navy  military  members.  This  instruction  covers  surveys  of  active  duty  and  reserve 
Navy  sailors... civilian  employees,  as  well  as  those  retired  from  active  duty,  reserve  or 
civilian  status.”  Like  the  instructions  for  the  NPS  IRB,  this  instruction  protects  Navy 
personnel  from  undue  influence  and  excessive  survey  burdens.  The  Navy  Survey 
Approval  Manager  is  the  Washington,  DC  liaison,  Navy  Survey  Approval  Manager  and 
Director  of  the  Institute  for  Organizational  Assessment  for  the  Navy  Personnel  Research, 
Studies,  and  Technology  Department  (NPRST/Pers-14).  In  most  cases,  to  obtain  survey 
approval  and  receive  an  OPNAV  Report  Control  Symbol  (RCS),  the  Navy  Survey 
Approval  Manager  must  determine  that  a  survey  has  never  been  performed  before  (i.e., 
that  the  proposed  survey  is  not  either  duplicating  a  prior  effort  or  that  the  data  being 
sought  is  available  via  other  sources),  that  it  is  endorsed  by  SES  level  sponsors,  and  that 
the  survey  is  being  conducted  in  a  technically  rigorous  manner  that  maximizes  the 
information  obtained  with  minimal  respondent  burden. 

However,  because  BUMEDINST  6220. 12B  states,  “NMCPHC  shall  develop 
methods  to  improve  and  facilitate  medical  surveillance,  response,  and  communication. 
This  includes  regular  program  evaluation  to  identify  areas  for  process  improvement,”  the 
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Navy  Survey  Approval  Manager  determined  that  only  BUMED  approval  was  required 
for  this  survey.  Thus,  the  approval  process  was  delegated  to  the  Forms  and  Reports 
Manager  for  BUMED  and  the  final  draft  of  the  survey  was  submitted  to  him.  “Given 
[that]  the  BUMED  Instruction  [supports  this  survey].  I'm  okay  with  considering  this 
fulfilling  a  BUMED  requirement  rather  than  a  Navy-wide  personnel  survey”  (P. 
Rosenfeld,  personal  communication,  October  6,  2010).  Upon  completion  of  the  Report 
Analysis  Data  form,  OPNAV  5214/10,  signed  by  CAPT  Clagget,  the  Director  of  the 
Preventive  Medicine  Department  at  the  NMCPHC,  the  survey  was  assigned  RCS 
BUMED  6220-3.  To  demonstrate  that  the  survey  was  officially  approved,  this  Report 
Control  System  number  was  displayed  to  recipients  of  the  notification  e-mails  via  the 
subject  line  and  it  was  displayed  on  the  title  line  of  the  survey  itself. 

B.  PRETESTING  THE  SURVEY 

Good  survey  practices  dictate  that  pre-testing  a  survey  helps  minimize  undue 
respondent  error,  which  can  be  done  in  several  ways.  “Stage  1:  Review  by 
knowledgeable  colleagues  and  analysts”  (Dillman  et  al.,  2008,  p.  140).  First,  I  had  my 
sponsor,  advisor,  and  the  ESSENCE  Program  Officer  review  the  questions  for  technical 
merit  and  question  readability.  They  also  looked  at  whether  all  the  necessary  questions 
were  included.  Since  pretesting  occurred  twice  during  the  design/fielding  phases,  a  more 
suitable  and  concrete  survey  was  developed  as  mentioned  in  the  previous  chapter. 

In  the  second  stage,  “interviews  to  evaluate  cognitive  and  motivational  qualifiers” 
were  conducted.  I  cognitively  interviewed  two  ESSENCE  monitors  over  the  phone. 
Two  of  the  interviews  were  conducted  with  health  professionals,  ranging  from  a  medium- 
to-highly  ranked  enlisted  health  professional,  to  a  medium-to-highly  ranked  officer.  The 
cognitive  interviews  allowed  me  to  gauge  whether  respondents  interpreted  all  the 
questions  in  the  manner  intended,  both  in  the  most  fundamental  sense  that  they 
understood  all  the  words,  as  well  as  that  they  understood  the  broader  intent  of  the 
question.  In  addition,  cognitive  interviews  are  useful  for  verifying  that  “all  the  questions 
[are]  interpreted  similarly  by  respondents”  (Dillman  et  al.,  2008,  p.  141),  and  that  the 
response  scales  are  appropriate,  understandable,  and  correct.  One  of  the  most  important 
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results  of  cognitive  interviews  is  to  ascertain  if  each  respondent  can  answer  each  question 
(Dillman  et  al.,  2008,  p.  146).  As  a  result  of  the  cognitive  interviews,  I  wrote  two 
different  surveys  because  the  first  iteration  contained  some  confusing  answers  and  did  not 
fully  represent  the  complete  ESSENCE  monitor  population. 

The  last  stage  of  pretesting  involved  asking  people  who  had  not  participated  in  the 
cognitive  interview  and  revision  process  to  take  the  survey.  A  complete  fielding  mock-up 
the  survey  was  given  to  them  just  as  it  would  have  been  provided  to  the  actual 
respondents.  In  this  final  iteration,  the  survey  was  pretested  by  two  Navy  ESSENCE 
users.  In  particular,  they  checked  to  see  if  I  “did  something  silly,”  as  Dillman  so 
adequately  stated  (Dillman  et  al.,  2008,  p.  147).  When  working  so  closely  on  a  project, 
anyone  can  loose  sight  of  the  goal,  and  these  pretesters  provided  the  sanity  check  the 
survey  needed  before  it  went  into  the  official  fielding  phase. 

C.  WEB-BASED  FIELDING 

The  survey  was  administered  using  SurveyMonkey,  a  commercial  Web-based 
survey  software  system.  The  advantages  of  administering  a  Web-based  survey  rather 
than  paper-based  survey  are  that  it  minimizes  cost,  as  well  as  the  ease  of  distributing 
notification  e-mails  more  than  once.  Web-based  surveys  also  tend  to  be  faster  than 
paper-based  survey  that  must  be  mailed  to  respondents  and  they  eliminate  data  entry.  In 
addition,  skip  logic  can  be  automated  within  the  Web-based  survey,  which  makes  it 
easier  and  faster  for  a  participant  to  take  the  survey. 

Due  to  the  transitory  nature  of  the  military,  the  Navy  and  Marine  Corps  monitors 
asked  to  participate  in  this  survey  may  have  transferred  jobs  or  billet  locations  since  the 
last  time  they  registered  for  an  ESSENCE  account.  Also,  since  active  and  disabled 
account  holders  were  included  on  this  list  of  possible  participants,  it  would  be  difficult  to 
track  down  the  location  of  each  member  when  the  survey  was  administered. 

E-mails  are  the  best  way  to  locate  service  members  regardless  of  their  location. 
Without  knowing  precisely  where  monitors  could  have  been  working  when  the  survey 
was  open  would  require  a  multitude  of  paper  surveys  sent  all  over  the  world  to  track 
down  each  monitor.  The  cost  of  stamps  and  envelopes  would  also  increase  as  well. 
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CAPT  Clagett  sent  out  a  pre-notification  e-mail,  discussed  below,  to  notify  the 
monitors  that  the  official  survey  would  be  sent  the  next  day.  If  the  survey  were  paper- 
based,  then  the  survey  would  have  had  to  wait  another  week  until  it  was  first  delivered. 
Furthermore,  disseminating  multiple  follow-up  letters  would  have  been  difficult  to 
accomplish  since  it  could  have  taken  weeks  before  respondents  returned  their  original 
survey.  The  Web-based  method  made  it  possible  to  send  four  follow-up  e-mails,  without 
sending  them  to  personnel  who  had  already  responded.  The  respondents’  answers  were 
collected  and  stored  the  day  they  completed  the  survey,  which  allowed  me  to  send  a 
follow-up  e-mail  that  included  responses  from  a  few  open-ended  questions. 

Lastly,  multiple  pages  of  this  survey  could  have  been  skipped  due  to  the  answer 
the  respondents  gave.  If  the  survey  were  administered  via  paper,  then  it  would  have 
looked  too  cumbersome  and  lengthy  for  the  monitor  to  complete.  Also,  the  ease  of 
having  me  put  the  skip  logic  together,  rather  than  having  the  participant  do  it,  would  have 
led  to  confusion  resulting  in  more  people  not  finishing  the  entire  survey. 

D.  RESPONSE  RATES 

The  survey  was  sent  to  225  Navy  and  Marine  Corps  users  with  either  an  active  or 
a  disabled  ESSENCE  account.  The  survey  pre-notification  e-mail  with  CAPT  Clagget's 
signature  block  and  return  e-mail  address  was  sent  on  November  8,  2010  to  all  potential 
respondents.  The  first  invitation  e-mail  was  sent  out  the  following  day,  asking  the 
monitors  to  take  the  survey.  Five  follow-up  e-mails  were  subsequently  sent  to 
nonrespondents  to  encourage  them  to  take  the  survey.  The  last  e-mail  was  sent  on 
December  6,  2010.  Thus,  the  survey  was  in  the  field  for  eight  weeks.  See  Appendix  C 
for  copies  of  all  respondent  correspondence. 

A  total  of  143  monitors  (64  percent)  responded  to  the  survey,  and  7  (0.03  percent) 
opted  out  of  taking  the  survey.  Figure  1  illustrates  the  percentage  of  participants  who 
responded  per  day  and  the  vertical  line  indicates  when  the  pre-notification,  initial 
notification,  and  follow-up  e-mails  were  sent. 
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Figure  1 .  The  percentage  of  respondents  who  replied  to  the  survey  while  it  was  open, 
from  November  8,  2010  until  December  30,  2010.  The  bubbled  areas 
indicated  the  dates  when  pre-notification,  notification,  and  follow-up  e- 
mails  were  sent.  The  black  line  shows  the  response  rate  over  time. 
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IV.  SURVEY  RESULTS 


This  chapter  discusses  the  survey  results.  Overall,  the  monitors  who  responded  to 
the  survey  were  generally  quite  happy  with  the  current  ESSENCE  system.  A  few 
individuals  suggested  how  to  better  the  system.  Furthermore,  most  of  the  respondents 
indicated  they  use  the  system  as  it  was  intended  to  be  used:  for  outbreak  detection  and 
situational  awareness.  However,  a  select  group  is  experiencing  or  has  experienced 
problems  with  the  system,  all  of  which  are  discussed  in  more  detail  below.  Complete 
survey  results,  including  all  open-ended  question  responses,  are  contained  in  Appendix 
D.  Appendix  D  also  includes  a  complete  list  of  survey  respondent  suggestions  for 
making  the  ESSENCE  system  easier  to  use  and  more  effective. 

A.  ASSESSING  MONITOR  USAGE 

The  survey  begins  by  asking  respondents  how  often  they  log  into  the  ESSENCE 
system.  As  shown  in  Figure  2,  across  all  types  of  users,  57  percent  log  in  routinely  (that 
is,  at  least  a  couple  of  times  per  week).  On  the  other  hand,  6  percent  of  the  respondents 
said  they  never  log  in  to  ESSENCE.  There  are  no  statistically  significant  differences  in 
login  frequency  by  rank,  professional  background,  support  facilities,  or  user  level. 


Q2:  On  average,  how  often  to  do  you  log  into  ESSENCE? 
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times  per  week)  times  per  month)  monthly  (a  couple 

times  per  year) 


Nevei 


N=142 


Figure  2.  Distribution  of  how  often  respondents  logged  into  ESSENCE. 
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When  asked  whether  they  log  into  ESSENCE  only  after  receiving  an  e-mail  alert, 
roughly  one  respondent  in  six  (13.4  percent)  said  yes.  As  shown  in  Figure  3,  this  type  of 
log  in  behavior  differs  by  rank  (%2  =  9.075,  p  =  0.0283), 3  where  100  percent  of  the 
contractors  said  they  only  logged  in  after  receiving  an  e-mail  alert,  followed  by  31 
percent  of  the  04-06s,  15  percent  of  all  enlisted  personnel,  and  none  of  the  01-03s  or 
civilians  said  they  logged  into  the  system  due  to  an  e-mailed  alert.  (Caution  is  warranted 
when  interpreting  the  E1-E3,  E7-E9,  and  a  contractor  category  as  the  total  number  of 
respondents  in  each  of  these  is  small.)  One  possible  explanation  for  this  result,  which 
cannot  be  confirmed  by  the  survey  data,  is  that  monitors  who  only  log  in  for  e-mail  alerts 
have  other  tasks  that  prevent  them  from  logging  into  ESSENCE  more  frequently. 

Of  the  1 1  respondents  who  only  log  in  when  they  receive  an  e-mail  alert,  eight  (73 
percent)  rarely  log  into  the  system  (that  is,  at  the  most  a  few  times  per  month)  and  when 
they  do  log  in,  it  is  primarily  because  they  received  either  a  yellow  or  red  alert.  Only  four 
respondents  (out  of  the  11)  actually  utilize  the  alert  module  to  explore  the  alerts  received 
in  their  inbox.  Furthermore,  eight  run  syndromic  or  reportable  diseases  queries.  Only  six 
of  the  1 1  respondents  investigate  alerts  and  five  use  ESSENCE  to  do  so.  A  possible 
cause  for  the  lack  of  ESSENCE  use  is  because  nine  out  of  10  of  these  respondents  have 
not  seen  an  actual  outbreak  using  ESSENCE.  Of  the  seven  respondents  who  rarely  log 
into  the  system  and  have  disabled  accounts,  this  could  be  attributed  to  ESSENCE  not 
functioning  they  way  they  intended:  by  providing  EED  and  SA. 


3  The  chi-squared  test  was  conducted  on  the  cross-tabulation  of  question  4  with  question  49  for  rank 
categories  E4-E6,  01-03,  04-06,  and  civilian.  The  rank  categories  for  E1-E3,  E7-E9,  and  contractor 
were  omitted  because  each  category  contained  less  than  10  respondents,  and  thus,  resulted  in  multiple  cells 
with  0  counts.  All  subsequent  chi-squared  tests  involving  rank  were  conducted  similarly. 
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Figure  3.  Percentage  of  respondents  who  log  into  ESSENCE  only  when  they  receive 
an  e-mail  alert  by  rank  category.  Overall,  roughly  80  percent  of  the 
monitors  do  not  log  in  just  because  they  receive  an  e-mail  alert. 


The  next  set  of  questions  asked  the  respondents  whether  they  ran  reportable 
disease  inquiries  while  logged  into  ESSENCE.  As  shown  in  Figure  4,  greater  than  one 
out  of  two  respondents  (53  percent)  indicated  that  they  ran  reportable  disease  queries 
routinely  (i.e.,  at  least  a  couple  of  times  a  week).  As  shown  in  Figure  5,  there  is  a 
statistically  significant  difference  (%  =  26.876,  p  =  0.0298)  by  rank  for  those  who 
conducted  reportable  diseased  queries,  with  El-E3s,  01-03s,  and  civilians  more  likely 
to  run  reportable  disease  inquiries  routinely.  This  is  consistent  with  the  fact  that 
reportable  diseases  are  primarily  used  in  conjunction  with  NDRSi,  and  to  confirm  that 
their  departments  had  submitted  their  medical  reportable  events  on  time,  which  are  tasks 
not  usually  done  by  mid-grade  officers  in  the  U.S.  Navy  and  Marine  Corps. 
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100% 


Q9:  On  average,  how  often  do  you  run  a  reportable  diseases  Query  ? 
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Figure  4.  Distribution  of  how  often  respondents  ran  a  reportable  disease  query. 


Figure  5.  Frequency  of  how  often  respondents  run  reportable  disease  queries  by  rank. 

Across  all  ranks,  35  percent  of  the  monitors  routinely  run  reportable  disease 

queries. 


In  addition,  there  is  a  statistical  difference  between  the  percentage  of  monitors 
with  an  NDRSi  account  by  user  level  (x2  =  6.809,  p  =  0.0332)4  and  by  professional 


4  User  levels  (question  45)  are  single  MTF,  multiple  MTF,  and  NEPMU  level  monitors. 


background  (%2  =  15.275,  p  =  0.0016). 5  In  Figure  6,  100  percent  of  the  NEPMU  level 
users  had  an  NDRSi  account,  while  26  percent  of  the  single  MTF  level  users  did  not. 
Furthermore,  as  shown  in  Figure  7,  all  of  the  physicians,  and  over  70  percent  of  the 
nurses,  did  not  have  an  NDRSi  account,  while  75  percent  or  more  of  the  other  types  of 
health  professionals  did  have  an  account.  Lastly,  these  same  ESSENCE  monitors  did  not 
verify  the  reportable  diseases  list  they  saw  in  ESSENCE  with  co-workers  nor  NDRSi 
because  they  did  not  have  accounts  and  did  not  require  them,  nor  will  they  in  the  future 
(Figure  8). 


Q10:  Do  you  have  a  Navy  Disease  Reporting  System,  internet 
(NDRSi)  account? 
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Figure  6.  By  user  level,  percentage  of  respondents  who  have  an  NDRSi  account. 


5  Professional  background  categories  (question  48)  are  preventive  medicine  technician,  environmental 
health  officer,  preventive  medicine  officer,  and  nurse,  corpsman,  epidemiologist,  and  physician.  The  latter 
three  categories  were  omitted  from  the  chi-squared  test  because  each  category  contained  less  than  10 
respondents,  and  thus,  resulted  in  cells  with  zero  counts.  All  subsequent  chi-squared  tests  involving  the 
professional  background  variable  were  conducted  similarly. 
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QIC:  Da  you  have  a  Navy  Disease  Reporting  System,  internet  (NDRSi)  account? 


Cuiptmitu  Preventive  Epidemu>t*p,iii  Environmental  Pieueulive  Nwse  Physician 

Medicine  Health  Office!  Medicine  OHiceis 

Technician 


Figure  7.  Percent  of  monitors  that  have  an  NDRSi  account  by  professional 

background.  An  average  of  80  percent  of  ESSENCE  users  have  an  NDRSi 

account. 


Oil:  if  confirmed,  do  von  nr  dn  you  with  a  co-worker  verify  that  the 
reportable  diseases  you  see  in  ESSENCE  are  also  inputed  into  NDRSi? 


Clinic  Naval  Hospital  Naval  Medical  Center  NEPML  N-9S 


Figure  8.  Percent  of  monitors  that  have  an  NDRSi  account  support  facility.  An 
average  of  94  percent  of  ESSENCE  users  confirm  that  the  reportable 
diseases  that  they  see  in  ESSENCE  are  also  put  into  NDRSi. 


As  shown  in  Figure  9,  36  percent  of  the  monitors  routinely  run  syndromic  queries, 
but  19  percent  never  use  the  module  and  14  percent  do  not  know  what  it  is.  Unlike 
reportable  diseases  queries,  there  is  no  difference  in  the  frequency  of  syndromic  queries 
by  demographic  groupings.  However,  the  way  the  respondents  used  the  data  details  while 
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running  syndromic  queries  are  statistically  different  (%2  =  26.699,  p  =  0.0085), 6  namely 
for  the  data  details  parameter  “age  range”  amongst  the  various  rank  categories  (Figure 
10).  One  possible  reason  for  this  result,  although  not  explainable  from  this  survey,  is  that 
age  range  is  not  always  important  when  narrowing  down  a  clustering  communicable 
disease  in  a  given  area.  For  instance,  if  an  outbreak  occurs  on  a  ship,  the  narrowing  down 
of  the  patients  age  range  does  not  help  the  investigator  track  the  outbreak,  while  the  other 
parameters  do;  for  instance,  the  clinics  they  visit  (close  to  the  docked  ship)  and  the 
patient  category  (active  duty  personnel).  Moreover,  if  the  monitor  deals  mostly  with 
active  duty  patients,  age  range  would  not  matter,  unlike  some  other  monitor  who  is 
looking  for  an  outbreak  amongst  children  or  the  elderly. 


:o°o 


o% 


Q14:  On  average,  how  often  do  you  run  a  syndromic  Query? 


n=27 


Weekly  (a  couple  Monthly  (a  couple  More  than  monthly  Never 

times  per  week)  times  per  month )  ( a  couple  times  per 

year) 


I  don't  know  what 
this  is 


Figure  9.  Distribution  of  how  often  respondents  ran  a  syndromic  query. 


6  The  chi-squared  test  was  conducted  on  the  cross-tabulation  of  question  16  with  question  49. 
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Q16:  When  you  run  syndromic  Query,  how  often  are  the  following 
parameters  important  to  your  investigation? 


E1-E3  E4  -  E6  E7  -  E9  01-03  04-06  Civilian  Contractor 

N=83 


Figure  10.  Frequency  of  how  often  monitors  use  the  age  range  parameter  during  an 
investigation.  An  average  of  42  percent  of  the  rank  categories  always  use 
age  range  while  investigation  a  particular  syndrome. 

In  terms  of  investigating  alerts,  75  percent  of  the  monitors  investigate  alerts 
(Figure  11),  and  approximately  nine  out  of  10  (89  percent)  of  those  use  ESSENCE  to 
conduct  their  investigations  (Figure  12).  There  is  a  statistically  significant  difference  (% 
=  8.875,  p  =  0.0029)7  by  user  level  of  those  who  use  the  system  during  their 
investigations.  The  majority  of  the  multiple  MTF  level  users  tend  to  use  ESSENCE, 
whereas  22  percent  of  the  single  MTF  level  users  and  1 1  percent  of  the  NEPMU  level 
users  do  not  use  the  system  at  all  (Figure  13).  One  possible  explanation  for  this 
phenomenon,  which  cannot  be  confirmed  by  the  survey  data,  is  that  ESSENCE  provides 
the  user  with  a  means  to  check  multiple  facilities  at  one  time,  even  the  entire  Navy  and 
Marine  Corps  (given  the  proper  permissions),  allowing  the  user  to  use  one  resource  vice 
multiple  external  tools.  ESSENCE  gives  the  account  holder  the  ability  to  observe  alerts 
or  potential  outbreaks  in  a  multitude  of  locations,  thereby  giving  the  monitor  a  starting 
point  for  an  investigation.  Single  MTF  users  are  more  intimately  familiar  with  the 
doctors  and  hospitals  within  their  clinic  or  hospital  and  can  speak  with  them  more  often, 
where  a  multiple  MTF  user  does  not  have  the  ability  to  speak  with  physicians  and  nurses 
from  all  the  hospitals  as  easily  during  their  investigations. 

7  The  chi-squared  test  was  conducted  on  the  cross-tabulation  of  question  20  with  question  45. 
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Figure  11.  Distribution  of  how  whether  respondents  investigate  alerts. 
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Figure  13.  Percentage  of  respondents  who  use  ESSENCE  to  conduct  investigations 

about  alerts.  Overall,  roughly  80  percent  of  the  monitors  do  use  ESSENCE 
to  conduct  their  investigations  of  a  potential  outbreak  alerted  to  them  via  the 

ESSENCE  system. 


There  is  a  statistical  difference  (yj  =  28.246,  p  =  0.005 1)8  by  support  facility  for 
those  who  use  FMP  during  an  investigation.  Illustrated  in  Figure  14,  it  is  more  likely  that 
clinic  and  Naval  Medical  Center  users  will  always  use  FMP,  while  NEPMU  users  are  the 
least  likely  to  use  this  data  detail  when  investigating  clustering  within  an  alert.  Over  60 
percent  of  the  E7-E9’s  and  04-06’s  always  use  FMP. 

Q23:  How  often  do  you  cheek  the  following  dalu  details  lo  look  for  clustering 
within  an  alert? 


Nevei 

Riiely 

■  Sometimes 

■  Frequently 
■Always 


N=7'1 

Figure  14.  Frequency  of  how  often  monitors  use  the  data  details,  FMP,  while 

investigating  clustering  within  an  alert. 

8  The  chi-squared  test  was  conducted  on  the  cross-tabulation  of  question  23D  with  question  46. 
Support  facilities  are  clinics,  naval  hospitals,  naval  medical  centers,  and  NEPMUs. 
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Another  data  details  functionality  that  was  significantly  different  (%  =  17.737,  p  = 
0.0383)9  by  rank  is  how  often  monitors  use  the  data  details,  MTF,  while  investigating 
alerts  (Figure  15).  One  possible  interpretation,  which  cannot  be  confirmed  by  the  survey 
data,  is  that  most  of  the  rank  categories  understand  how  to  make  a  site  selection  list  and 
use  that  list  when  running  a  query  to  investigate  alerts  in  their  area.  Rather  than  select  a 
particular  MTF  during  the  query,  they  already  specified  their  location  by  selecting  the 
location  through  the  “site  selection”  feature  that  limits  their  area  of  search  during  a 
syndromic  query  or  within  the  alert  list  module. 


Q23:  How  often  do  you  check  the  following  data  details  to  look  for  clustering 

within  an  alert? 


V-R4 

Figure  15.  Frequency  of  how  often  monitors  use  the  data  details  MTF  when  looking  for 

clustering  within  an  alert  by  rank. 

There  are  also  significant  differences  by  user  level  (y2  =  33.993,  p  =  <0.0001), 10 
support  facility  (y2  =  38.181,  p  =  <0.0001), 11  and  professional  backgrounds  (x  =  30.278, 
p  =  0.0002) 12  with  how  often  AHLTA  is  used  during  an  investigation.  Approximately 
three  out  of  five  (57  percent)  of  NEPMU  level  monitors  rarely,  if  ever,  use  AHLTA  as  an 
external  tool  to  aid  them  in  their  alert  investigations  (Figure  16).  Furthermore,  an 
average  of  two  out  of  three  clinic,  naval  hospital,  and  naval  medical  center  monitors 

9  The  chi-squared  test  was  conducted  on  the  cross-tabulation  of  question  23H  with  question  49. 

19  The  chi-squared  test  was  conducted  on  the  cross-tabulation  of  question  24A  with  question  45. 

1 1  The  chi-squared  test  was  conducted  on  the  cross-tabulation  of  question  24A  with  question  46. 

12  The  chi-squared  test  was  conducted  on  the  cross-tabulation  of  question  24A  with  question  48. 
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always  use  AHLTA  to  assist  them  in  their  investigations  (Figure  17).  One  possible 
explanation  for  this  result  is  that  NEMPU  level  users  and  supporters  do  not  have  access 
to  AHLTA  and  do  not  need  it.  NEMPU  monitors  are  regional  investigators  who  do  not 
need  to  examine  alerts  with  extreme  detail,  unlike  the  clinics,  NH,  and  NMC  users 
because  they  are  examining  their  own  area  and  part  of  their  departmental  duties,  which  is 
overseen  by  the  regional  staff  members  at  the  NEMPU  level.  Figure  18  illustrates  that  85 
percent  of  the  preventive  medicine  technicians  always  use  ALHTA,  while  50  percent  of 
the  physicians  and  environmental  health  officers  do  not.  Part  of  preventive  medicine 
technicians’  duties  is  to  investigate  alerts  and  use  whatever  means  necessary  to  do  so. 
AHLTA  enables  them  to  obtain  the  details  of  patient  information  and  why  they  have  been 
assigned  a  specific  ICD-9  code.  If  the  ICD-9  is  related  to  their  investigation,  then  they 
can  proceed  towards  determining  if  an  actual  outbreak  is  occurring.  In  comparison, 
corpsman  may  not  have  access  to  AHLTA  and  physicians  may  have  already  assigned  a 
particular  ICD-9  code  to  their  patients  or  have  spoken  with  fellow  physicians,  and 
therefore,  do  not  need  to  verify  this  information  in  AHLTA. 


Figure  16.  Frequency,  by  user  level,  of  how  often  respondents  use  AHLTA  as  an 

external  aid  while  conducting  alert  investigations. 
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Q24:  How  often  do  you  use  the  following  external  tools  to  further 
investigate  clustering  within  an  alert? 


Clinic  Naval  Hospital  Naval  Medical  Center  NFPM1J  N=80 


Figure  17.  Frequency,  by  support  facility,  of  how  often  respondents  use  AHLTA  as  an 

external  aid,  while  conducting  alert  investigations. 


Q24:  How  often  do  you  use  the  following  external  tools  to  further  investigate  clustering 

within  an  alert? 


( orpsimn  Preventive  Kpufcmioloyisi  I'nviTonmeniJil  Preventive  Nurse  Physician 

Medicine  Health  Officer  Medicine 

Technician  Officers 


Never 

Raich 

■  Sometimes 

■  hicqucnlly 

■  Always 


N-87 


Figure  18.  Frequency,  by  professional  background,  of  how  often  respondents  use 
AHLTA  as  an  external  aid,  while  conducting  alert  investigations. 


B.  ASSESSING  PERCEIVED  VALUE 

I  first  begin  by  displaying  the  results  of  how  often  respondents  applied  for  their 
current  ESSENCE  account,  and  how  often  they  spoke  with  representatives  while  in  the 
process  of  obtaining  it.  This  is  important  due  to  respondent  feedback  through  the  open- 
ended  comments  sections  of  this  survey.  Distaste  for  constant  password  updates  and 
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account  requirement  are  prevalent  throughout  the  comment  sections  and  the  distributions 
of  survey  results  are  not  statistically  different  by  user  level,  rank,  professional 
background,  and  support  facility.  Due  to  an  increase  in  operational  tempo  within  the 
Department  of  Defense,  military  members  are  required  to  leave  their  billets  often  and 
must  reacquire  new  accounts  after  30  days  if  the  program  has  not  been  logged  into. 

In  Figures  19-21,  questions  35-37  illustrate  the  distribution  of  respondents  who 
applied  more  than  once  for  their  current  ESSENCE  account,  had  to  speak  with  an 
ESSENCE  help  desk  representative,  and  the  frequency  of  phone  calls/e-mail  needed  to 
acquire  the  account.  About  40  percent  of  users  must  apply  more  than  once.  On  average 
one  out  of  two  (48  percent)  had  to  speak  with  a  help  desk  representative  while  waiting  to 
access  their  current  account.  Finally,  45  percent  contacted  the  help  desk  more  than  three 
times.  In  summary,  users  report  what  seem  to  be  significant  difficulties  in  acquiring  an 
ESSENCE  account;  difficulties  that  surely  do  not  engender  feelings  of  good  will  toward, 
nor  confidence  in,  the  system. 


Q35:  Did  you  have  to  apply  more  than  once  to  obtain  your 
current  ESSENCE  account? 

100% 

80% 


40% 

20% 

0% 


Figure  19.  Distribution  of  respondents  who  had  to  apply  more  than  once  to  obtain  their 

current  ESSENCE  account. 
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Figure  20.  Distribution  of  respondents  who  had  to  speak  with  an  ESSENCE  helpdesk 

representative  while  waiting  to  access  their  current  ESSENCE  account. 


100% 


Q37:  Estimate  how  many  times  you  needed  to  contact  the  helpdesk  (via  e-mail 
or  phone  call)  to  acquire  your  current  ESSENCE  account. 


80% 

60% 


40% 

20% 

0% 


n-21 


1  turn  2  timei  3  times  4  times  5  or  more  times 


n=5 


I  con’t  remember 

N=133 


Figure  2 1 .  Distribution  of  how  many  times  respondents  had  to  contact  the  helpdesk  to 

acquire  their  current  ESSENCE  account. 


After  examining  the  data  to  determine  respondents’  perceived  value  of  the  system, 
I  discovered  that  overall,  the  monitors  believe  that  the  ESSENCE  system  performs  its 
situational  awareness  role  well,  but  they  find  it  lacking  in  its  early  event  detection  role. 
For  example,  Figure  22  shows  that  three  out  of  four  respondents  have  not  discovered  an 
actual  outbreak  using  ESSENCE.  There  is  a  statistically  significant  difference  (x  = 
12.363,  p  =  0.0062)13  by  rank  of  monitors  who  discovered  an  actual  outbreak  using  the 


13  The  chi-squared  test  was  conducted  on  the  cross-tabulation  of  question  29  with  question  48. 
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biosurveillance  system.  Over  60  percent  of  the  rank  categories  except  01-03s  have  not 
used  ESSENCE  for  early  event  detection  (Figure  23).  The  interpretation  is  that  the 
majority  of  the  respondents  believe  they  did  not  see  an  actual  outbreak  occur  although 
ESSENCE  indicated  that  one  existed.  Further  proof  that  respondents  tend  to  see  more 
false  positives  exists  when  a  cross-tabulation  of  question  29  was  conducted  with  question 
33H,  which  asks  respondents  if  they  agree  or  disagree  with  the  statement:  “I  find  that  the 
alerts  in  ESSENCE  are  often  false-positives.”  Nine  out  of  10  monitors  (Figure  24)  who 
said  they  have  not  discovered  an  actual  outbreak  said  they  agree  or  are  neutral  to  this 
statement.  A  few  respondents  who  have  not  discovered  an  actual  outbreak  using 
ESSENCE  said  they  would  like  “training  and  insight  into  how  syndromic  alerts  are 
derived”  and  “how  to  detect  false  positives.”  Another  monitor  stated  that  “understanding 
how  the  alert  functions  would  be  better  [to  know].  Right  now,  I  feel  that  the  alerts  are  set 
to  be  too  sensitive,  as  we  get  quite  a  few  of  them  here.  Investigating  all  of  them  would 
take  far  too  much  time.  When  we  do  investigate  them,  we  never  come  to  a  conclusion  on 
the  cause  (if  any)  for  the  increase  in  patients  for  that  particular  syndrome.”  Finally,  one 
user  said  that  they  would  like  to  “  have  [a]  person  [who  is]  doing  the  [application] 
screening  [to]  ask  for  all  the  additional  info  needed  at  once  instead  of  responding  (a  week 
later)  after  one  item  has  been  submitted  then  asking  for  something  else.” 


Q29:  Have  you  discovered  an  actual  outbreak  using  ESSENCE? 

100% 

80% 

60% 

40% 

20% 

0% 

Figure  22.  Distribution  of  respondents  who  say  they  have  discovered  an  actual 

outbreak  using  ESSENCE. 


n=99 


Yes  No 
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Q29:  Have  you  discovered  an  actual  outbreak  using  ESSENCE? 

iou%  - - - — - 


HI -HI  14-  lift  K7-K9  01-01  04 -Oft  Civilian  (‘(infractor  n  120 


Figure  23.  Percentage  of  monitors,  by,  rank,  who  have  discovered  an  actual  outbreak 
using  ESSENCE.  Roughly  75  percent  of  the  respondents  said  they  have  not 
seen  an  actual  outbreak  in  ESSENCE. 


Figure  24.  Percentage  of  monitors  who  have  not  discovered  an  actual  outbreak  using 
ESSENCE  but  who  also  either  are  neutral  or  agree  with  the  statement  that 
they  find  that  ESSENCE  alerts  are  usually  false-positives. 


Of  the  33  respondents  who  have  observed  an  actual  outbreak  in  ESSENCE,  over 
20  confirmed  actual  cases  of  ILI,  GI,  and  respiratory  outbreak  types  as  well,  seven  have 
seen  heat,  febrile,  neurological,  pertussis,  and  exposure  to  chemical/toxins  cases. 
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However,  75  percent  of  the  respondents  observe  miscoded  ICD-9  codes  in 
ESSENCE  (Figure  25).  Miscoding  causes  alerts  to  increase  because  providers, 
immunizations  representatives,  MTF  coding  representatives,  or  nurses  have  input  the  data 
incorrectly  into  the  system  and  currently  these  mistakes  cannot  be  removed  or  updated  by 
monitors  who  have  verified  the  miscodings  using  AHLTA  or  CHCS.  Of  the  100 
respondents  who  witness  miscodings,  73  percent  agree  that  it  would  be  easier  to  use 
ESSENCE  if  a  patient’s  final  diagnosis  was  viewed  differently  in  the  system,  perhaps  if 
bolded.  In  addition,  95  percent  agree  that  it  would  be  easier  to  use  ESSENCE  if  a 
patient’s  miscoded  ICD-9  codes  were  updated  and  correctly  input  into  the  system.  Again, 
miscodings  are  a  potentially  significant  cause  of  the  false-positives  within  the  system. 


Figure  25.  Distribution  of  respondents  who  have  observed  a  miscoded  ICD-9  code  in 

ESSENCE. 

Question  33  asks  respondents  whether  they  agree  or  disagree  with  eight 
statements  regarding  the  ESSENCE  program,  presented  in  Figure  26.  An  average  of  92 
percent  of  the  respondents  said  they  value  the  system;  that  is,  they  tend  to  strongly  agree, 
agree,  or  are  neutral  towards  all  eight  statements.  However,  it  is  important  to  understand 
why  the  other  8  percent  do  not  value  the  system  and  what  can  be  done  in  the  future  to 
remedy  their  perceptions,  issues,  or  problems  in  order  to  make  the  system  valuable  to  all 
future  users. 
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Figure  26.  Distribution  of  responses  regarding  respondents  perceived  value  of  the 

ESSENCE  system. 
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Roughly  85  percent  of  the  monitors  feel  comfortable  using  the  ESSENCE 
program.  On  the  other  hand,  over  half  of  the  15  percent  of  respondents  who  do  not  feel 
comfortable  with  the  system  believe  that  ESSENCE  does  not  provide  them  with  SA 
about  communicable  diseases  in  their  area  and  they  do  not  think  that  bolding  a  patient’s 
final  diagnosis  would  make  ESSENCE  easier  for  them  to  use.  Of  these  20  individuals, 
70  percent  log  in  rarely  (that  is,  monthly  to  never),  50  percent  are  multiple  MTF  level 
users,  roughly  40  percent  are  preventive  medicine  technicians  ranging  in  rank  between 
E4-E9  and  are  split  between  having  active  and  disabled  ESSENCE  accounts. 

Figure  27  indicates  there  is  a  statistical  difference  (%2  =  16.107,  p  =  0.0409)14  by 
user  level  of  those  who  think  ESSENCE  provides  their  department  with  situational 
awareness  about  communicable  diseases  of  interest  in  their  area.  Twenty-seven  percent 
of  the  NEPMU  level  users  disagree  with  this  statement,  while  90  percent  of  the  MTF 
level  users  agree  that  ESSENCE  provides  them  with  a  suitable  amount  of  situational 
awareness  about  outbreaks  in  their  area.  The  chi-squared  test  conducted  on  the  cross- 
tabulation  between  this  statement  and  rank,  a  statistically  significant  result  (%  =  25.873,  p 
=  0.0112)  indicates  that  24  percent  of  the  01-03s  and  10  percent  of  the  04-06s  do  not 
feel  that  ESSENCE  provides  their  department  with  situational  awareness  either,  as 
depicted  in  Figure  28. 


Figure  27.  Percentage  of  monitors  by  user  level  who  agreed,  was  neutral,  or  disagreed 
with  the  statement  that  ESSENCE  provides  their  department  with  situational 
awareness  about  communicable  diseases  of  interest  in  their  area. 


14  The  chi-squared  test  was  conducted  on  the  cross-tabulation  of  question  33B  with  question  45. 
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Figure  28.  Percentage  of  monitors  by  rank  who  agreed,  was  neutral,  or  disagreed  with 
the  statement  that  ESSENCE  provides  their  department  with  situational 
awareness  about  communicable  diseases  of  interest  in  their  area. 


Similarly,  monitoring  trends  of  diseases  in  a  certain  location  over  time  allows  a 
monitor  to  track  and  perhaps  understand  the  local  trends  in  the  diseases.  There  is  a 
statistically  significant  difference  by  support  facility  (%2  =  29.292,  p  =  0.0036) 15  and  rank 
(%2  =  26.887,  p  =  0.0081)16  of  those  who  do  not  believe  that  ESSENCE  allows  them  to 
monitor  trends  of  diseases  over  time,  which  provides  the  situational  awareness  that  is  a 
key  role  for  ESSENCE.  Similar  to  the  results  in  Figure  27,  Figure  29  shows  that  20 
percent  of  the  NEPMU  support  facilities  do  not  think  ESSENCE  allows  them  to  monitor 
the  trends  of  diseases  in  their  area  over  time.  Furthermore,  17  percent  of  the  04-06s  and 
small  number  of  01-03s  and  civilians  believe  that  this  ability  is  lacking  within  the 
program  (Figure  30).  On  the  other  hand,  seven  out  of  10  users  think  ESSENCE  is  useful 
for  recognizing  trends  over  time,  and  believe  SA  is  being  provided  to  them. 


15  The  chi-squared  test  was  conducted  on  the  cross-tabulation  of  question  33C  with  question  46. 

16  The  chi-squared  test  was  conducted  on  the  cross-tabulation  of  question  33C  with  question  49. 
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Q33:  IIow  much  do  you  agree  or  disagree  with  the  following  statements 
regarding  Che  ESSENCE  program 


1(X)% 


Clinic  Naval  Hospital  Naval  Medical  ( 'enlct  NFPMIJ 


■  Agiee 

■  Neutral 
Disagiec 


N=1 18 


Figure  29.  Percentage  of  monitors  by  support  facility  who  agreed,  were  neutral,  or 
disagreed  with  the  statement  that  ESSENCE  allows  them  to  monitor  the 
trends  of  diseases  in  their  area  over  time. 


Figure  30.  Percentage  of  monitors  by  rank  who  agreed,  was  neutral,  or  disagreed  with 
the  statement  that  ESSENCE  allows  them  to  monitor  the  trends  of  diseases 

in  their  area  over  time. 
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In  addition,  there  was  a  statistical  difference  (%2  =  16.569,  p  =  0.0349)17  by  user 
level  or  those  who  believe  that  ESSENCE  allows  them  to  follow  particular  reportable 
diseases  in  their  area.  Once  more,  NEMPU  level  users  disagree  that  they  are  able  to 
follow  reportable  diseases  in  their  area  (Figure  31). 


Q33:  Eloiv  iiiiioh  do  you  iigrvi1  or  clisttgrco  mill  I  lie  following  statements 
regarding  l lie  ESSENCE  program 


1  Agitt 


Disagree 


Smple  MTF  level  user 


Mn tuple  MTF's  level  tisci 


NfcPMU  level  tisei 


Figure  31.  Percentage  of  monitors  by  user  level  who  believe  that  ESSENCE  allows 
them  to  follow  particular  reportable  diseases  in  their  area. 


Through  exhaustive  research  of  the  data,  two  questions  in  this  question  that 
identify  outbreaks  and  false-positives  are  the  most  important.  Inherently,  ESSENCE  is 
an  outbreak  detection  tool,  mostly  for  EED,  and  without  collaborating  programs,  false- 
positives  are  time  consuming  for  monitors  to  investigate.  Of  the  1 1  respondents  who  do 
not  think  that  the  time-series  module  is  useful  for  identifying  outbreaks,  each  one  did  not 
see  an  actual  outbreak  in  ESSENCE.  Roughly  70  percent  of  these  monitors  are  routine 
users  of  the  program,  54  percent  routinely  use  the  reportable  diseases  query  module  and 
27  percent  routinely  use  the  syndromic  query  module  as  well.  Six  respondents  use 
ESSENCE  to  investigate  alerts  and  about  50  percent  perceive  the  system  to  be  ineffective 
and  not  valuable.  Some  reported  that  they  use  “AHLTA,  CHCS,  and  labs”  more  often 
than  ESSENCE,  and  their  biggest  headache  comes  from  gaining  “access”  to  the  program 


17  The  chi-squared  test  was  conducted  on  the  cross-tabulation  of  question  33D  with  question  45. 
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in  the  first  place.  They  would  prefer  that  “ESSENCE  communicate  with  programs,  such 
as  AHLTA  and  CHCS,”  so  they  would  not  have  to  investigate  false-positives  and  one 
monitor  did  not  have  “access  to  Personal  Health  Information  (PHI)  information,” 
although  access  was  requested  multiple  times.  Finally,  users  reported  that  they  could  use 
ESSENCE  for  “situational  awareness,  but  not  for  outbreak  detection”  and  it  was  “not 
useful  during  the  H1N1  outbreak.” 

Lastly,  62  respondents  said  they  find  that  alerts  in  ESSENCE  have  a  propensity  to 
be  false-positives.  Sixty-six  percent  are  routine  login  users,  62  percent  run  reportable 
diseases  queries  and  50  percent  run  syndromic  queries.  Approximately  77  percent  use 
ESSENCE  to  investigate  alerts,  but  only  33  percent  has  detected  an  actual  outbreak  using 
the  system.  A  few  have  requested  “more  training,”  primarily  “hands  on  training  that 
provides  practical  use  in  one’s  department,”  such  as  “describing  how  [the]  alert  function 
works,”  outbreak  scenarios  and  times-series  module  analysis.  Some  respondents  recalled 
that  ESSENCE  “times  out  too  quickly”  and  constant  password  updating  makes  the 
account  holder  spend  more  time  speaking  with  ESSENCE  help  desk  representatives, 
rather  than  logging  in  and  checking  for  alerts.  Finally,  some  feel  that  the  “daily 
requirement  to  check  ESSENCE  seems  to  be  a  bit  much.”  One  respondent  commented, 

I  find  that  many  times  the  alerts  are  false  alerts  and  that  the  “syndrome” 
parameters  are  so  wide  that  sometimes  the  alert  doesn't  seem  credible. 

(i.e.,  a  rash  in  two  infants  will  trigger  an  alert  and  so  will  ptosis  of  the 
eyelid  diagnosed  in  the  optometry  clinic.)  I  will  check  ESSENCE  and 
then  check  the  AHLTA  encounter  and  contact  the  providers  or  clinics  to 
[obtain]  “ground  truth”  the  data  to  see  if  what  I  see  in  ESSENCE  is  truly 
what  is  happening  in  the  clinics.  I  do  use  ESSENCE  to  see  how  we  are 
doing  with  regard  to  trends  in  coding  for  certain  things  like  ILI,  Resp,  and 
GI  syndromes.  I  share  this  with  providers  as  appropriate.  I  also  check  the 
reportable  disease  section  and  use  this  as  a  “safety  net”  to  catch  any 
reportable  diseases  that  weren't  consulted  to  Prev  Med  or  didn't  show  up 
as  a  positive  lab  in  our  box  in  the  Laboratory.  We  follow  up  on  each 
reportable  disease  that  shows  up  in  ESSENCE.  At  times  we  will  find 
STDs  and  other  reportable  diseases  this  way. 

In  conclusion,  at  least  nine  out  of  10  respondents  value  ESSENCE,  but  the  8 
percent  who  do  not  value  the  system  have  important  recommendations  and  reasons  why 
they  do  not  favor  the  system.  Most  of  them  center  on  better  and  specific  training, 
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constant  password  renewal,  false-positives  that  decrease  actual  outbreak  detection, 
miscodings,  and  compatibility  with  other  external  systems,  such  as  AHLTA  and  CHCS, 
to  view  patient  information  while  logged  in. 

C.  ASSESSING  USER  TRAINING 

A  series  of  questions  were  developed  to  examine  how  ESSENCE  monitors  were 
trained  to  use  the  system  and  what  they  would  like  to  see  in  the  future.  There  are  no 
statistically  significant  differences  in  how  the  respondents  where  trained  by  user  level, 
support  facility,  professional  background.  Seventy  percent  of  the  respondents  were  self- 
taught,  followed  by  34  percent  who  were  trained  by  their  processors  or  downloaded  the 
ESSENCE  training  modules  and  tutorials,  and  21  percent  were  taught  by  their  bosses 
(Figure  32). 18 


Q38:  \\  ho  did  you  receive  your  ESSENCE  training  from? 
Check  all  (hat  uppl\. 


My  prpdecrsvor  My  boss  I  am  self  taught  I  downloaded  the  I  attended  a  conference  I  was  taught  by  the 

tiitNCt  training  where  l  was  taught  how  Ntl'MU 

module*  and  tutorials  to  work  tl>£  System 

N-126 


Figure  32.  Distribution  of  where  respondents  received  their  ESSENCE  training. 


With  no  statistical  difference  (by  rank,  user  level,  professional  background,  and 
support  facility),  the  majority  of  the  monitors  would  like  to  see  future  training  provided 
to  them  at  their  desks,  in  public  health  conferences,  in  hospital/clinic  meetings,  “C” 
school,  NEPMU,  and  CME  training.  However,  30  percent  do  not  want  to  have  meetings 

18  Percentages  add  up  to  more  than  100  percent  because  respondents  could  select  more  than  one 
response. 
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at  the  hospitals/clinic  meetings,  indicating  (not  verifiable  through  survey  data)  that 
meetings  during  working  hours  are  not  conducive  to  ESSENCE  training  and  should  be 
conducted  elsewhere.  Fewer  than  20  percent  of  users  do  not  want  to  see  training  at  their 
desks  or  in  health  conferences,  which  is  evidence  enough  that  these  would  be  better 
venues  for  ESSENCE  training. 

Ninety-seven  percent  of  the  respondents  reported  that  they  would  prefer  hands-on 
training.  There  were  no  significant  differences  by  rank,  professional  background,  user 
level,  or  support  facility.  Furthermore,  at  least  nine  out  of  10  monitors  would  like  to 
receive  initial  ESSENCE  user  training,  including  training  for  site  selection  set  up,  how  to 
evaluate  alerts,  query  module  applications,  and  outbreak  scenarios.  A  few  individuals 
also  wrote  that  they  would  like  to  learn  how  to  “detect  probable  false  positives,”  “use  the 
matrix  function,”  and  “time-series  explanation  and  14-day  incidence  explanations.” 


Q39:  Where  would  you  like  to  see  training  performed  for  current 
and  future  users  of  the  ESSENCE  system? 


1 01)% 


At  my  own  desk  In  public  health  In  hospital/clinic  In  "C"  school 

conferences  meetings 

N=131 


Figure  33.  Distribution  of  where  respondents  would  like  to  receive  their  ESSENCE 

training  in  the  future. 
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100% 

80% 

60% 

40% 

20% 

0% 


Q40:  How  would  you  like  to  see  training  provided  for  current 
and  future  users  of  the  ESSENCE  system? 


No 

■Yes 


Audio  and  video  based  Howerpoint  presentations  Hands-on  training  ^  ^ 


Figure  34.  Distribution  of  how  respondents  would  like  to  receive  their  ESSENCE 

training  in  the  future. 


Q41:  What  type  of  training  would  you  like  for  current  and 
future  users  of  the  ESSENCE  system? 

100% 

80% 

60% 

■No 

■Yes 

40% 

20% 


Initial  training  to  Site  selection  set  How  to  evaluate  Query  module  Outbreak  scenarios 
use  ESSENCE  up  alerts  applications 

N=132 


Figure  35.  Distribution  of  what  respondents  would  like  to  see  in  ESSENCE  training  in 

the  future. 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  SUMMARY  AND  CONCLUSIONS 

The  Navy  and  Marine  Corps  use  the  ESSENCE  system  for  early  detection  of 
diseases  and  other  public  health  threats  to  the  force  and  for  situational  awareness  on  the 
location  and  spread  of  such  diseases.  DoD  Directive  6490.2E  dated  August  24,  2009 
requires  that  “the  DoD  components  shall  conduct  comprehensive,  continuous,  and 
consistent  military  health  surveillance  to  implement  early  intervention  and  control 
strategies,  using  joint  technologies,  practices,  and  procedures  in  a  manner  consistent 
across  the  military  services.  Relevant,  timely,  actionable,  comprehensive  health 
surveillance  information  shall  be  collected  and  maintained  to  support  the  Armed  Force.” 
Following  this  guidance  in  2009,  the  U.S.  Navy  and  the  Navy  Bureau  of  Medicine  and 
Surgery  (BUMED)  mandated  that  the  “monitoring  of  ESSENCE  alerts  applies  to  all 
Navy  medical  treatment  facilities  (MTFs).  Health  surveillance  shall  be  conducted  to 
enable  early  intervention  and  control  strategies.  Monitoring  for  and  timely  reporting  of 
significant  health  events  that  may  adversely  affect  mission  accomplishment  and  shape  the 
commander’s  decision  making  is  vital  to  military  medicine  functions. 

Navy  ESSENCE  monitors  are  health  professionals,  including  preventive  medicine 
technicians,  doctors,  nurses,  public  health  professionals,  and  epidemiologists.  The 
agencies  that  monitor  ESSENCE  are  the  preventive  medicine  and  public  health 
departments  of  the  Navy,  which  are  spread  throughout  the  world.  They  are  comprised  of 
naval  clinics,  naval  hospitals,  and  naval  medical  centers.  The  Navy  Environment 
Preventive  Medicine  Units  (NEPMU),  which  are  located  in  San  Diego,  California,  Pearl 
Harbor,  Hawaii,  and  Norfolk,  Virginia,  also  monitor  disease  activity  using  ESSENCE. 
Finally,  the  Navy  and  Marine  Corps  Public  Health  Center  (NMCPHC)  oversees  patient 
syndromic  surveillance  throughout  the  Navy  as  mandated  by  BUMEDINST  6220. 12B. 

This  survey,  conducted  on  behalf  of  the  NMCPHC,  was  sent  to  225  Navy  and 
Marine  Corps  users  with  either  an  active  or  a  disabled  ESSENCE  account.  The  survey 
was  in  the  field  for  eight  weeks.  A  total  of  143  monitors  (64  percent)  responded  to  the 
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survey  and  seven  (3  percent)  opted  out  of  taking  it.  Survey  findings  include  that,  overall, 
92  percent  of  ESSENCE  account  holders,  past  and  present,  favor  using  the  system,  find  it 
valuable,  and  believe  the  training  they  received  was  adequate.  However,  users  raised 
four  main  issues:  1)  it  takes  an  excessive  amount  of  time  to  obtain  an  account,  2) 
passwords  are  required  to  be  changed  too  often,  3)  there  are  too  many  miscodings  leading 
to  excessive  false  positive  signals,  and  4)  training  and  training  tools  are  insufficient. 

First,  acquiring  an  account  and  maintaining  it  is  a  recurrent  problem  among  the 
users.  Also,  once  the  account  has  been  given,  users  must  change  their  passwords  every 
other  month.  One  respondent  took  the  time  to  follow-up  in  an  e-mail  and  wrote  that: 

the  stringency  of  the  15  digit  complex  password  that  has  to  be  re-set  every 
60  days.  I  find  myself  spending  an  incredible  amount  of  time  “trying  out” 
new  passwords. ..only  to  discover  it  may  have  failed  for  lack  of  a  second 
numeral,  etc.. ..and  then  have  to  do  it  all  over  again  in  60  days.  I  do  have  a 
few  “tricks”  that  help,  but  still  haven't  found  a  foolproof  way  to  satisfy  the 
requirement  in  a  time-effective  manner.  I  understand  the  need  for 
security —  perhaps  re-setting  the  pw  every  90  days  might  be  a  reasonable 
compromise  between  maintaining  a  high-level  of  security  and  trying  to 
increase  user  friendliness. 

Another  respondent  commented  in  the  survey: 

Don't  ask  for  me  to  change  my  password  as  often  as  requested.  I  have 
been  on  leave  or  TAD  twice  when  my  account  has  been  inactivated. 
Although  I  should  have  completed  the  task  before  I  left,  I  don't  change  my 
bank  info  as  much  as  I  have  to  change  ESSENCE.  I  understand  the 
privacy  issue,  but  most  medical  folks  not  involved  in  Preventive  Medicine 
wouldn't  even  know  where  to  access  ESSENCE  b/c  they  don't  know  what 
it  is.  I  think  it's  a  secure  site,  but  I  have  had  to  reset  my  password  several 
times. 

Understanding  that  safeguarding  of  personally  identifiable  information  (PII)  is 
very  important,  but  a  single  standard  should  be  in  place  for  all  programs  with  PII, 
meaning  that  password  change  requirements  should  be  the  same  length  as  AHLTA  or 
CHCS.  If  passwords  need  to  be  revised  often,  then  is  it  important  to  notify  the  users  of 
this  and  the  reasons.  A  well-informed  account  holder  is  a  happier  one. 

Second,  numerous  comments  declared  that  miscodings  are  too  prevalent  and 
respondents  believe  they  contribute  to  a  high  false  positive  rate  in  ESSENCE.  One 
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monitor  states,  “there  are  [too]  many  false  positives,  and  many  of  the  abbreviations 
remain  a  mystery.”  The  user  suggests,  “a  key  or  legend  would  help,”  and  if  one  exists,  to 
“make  it  more  apparent  to  access.  The  idea  of  the  program  is  marvelous  —  so  if  it  was  a 
little  easier  to  use  I  think  more  of  us  would  use  it  as  it  was  intended.”  Another  well- 
informed  user,  but  one  that  believes  false  positives  are  numerous,  said  that: 

The  daily  requirement  to  check  ESSENCE  seems  to  be  a  bit  much.  I  find 
that  many  times  the  alerts  are  false  alerts  and  that  the  “syndrome” 
parameters  are  so  wide  that  sometimes  the  alert  doesn't  seem  credible. 

(i.e.,  a  rash  in  2  infants  will  trigger  an  alert  and  so  will  ptosis  of  the  eyelid 
diagnosed  in  the  optometry  clinic.)  I  will  check  ESSENCE  and  then 
check  the  AHLTA  encounter  and  contact  the  providers  or  clinics  to 
'ground  truth'  the  data  to  see  if  what  I  see  in  ESSENCE  is  truly  what  is 
happening  in  the  clinics.  I  do  use  ESSENCE  to  see  how  we  are  doing  with 
regard  to  trends  in  coding  for  certain  things  like  ILI,  Resp,  and  GI 
syndromes.  I  share  this  with  providers  as  appropriate.  I  also  check  the 
reportable  disease  section  and  use  this  as  a  'safety  net'  to  catch  any 
reportable  diseases  that  weren't  consulted  to  Prev  Med  or  didn't  show  up 
as  a  positive  lab  in  our  box  in  the  Laboratory.  We  follow  up  on  each 
reportable  disease  that  shows  up  in  ESSENCE.  At  times,  we  will  find 
STDs  and  other  reportable  diseases  this  way.  (Sometimes  we  find  that  they 
are  simply  rule-outs  or  coding  errors,  i.e.,  'meningitis'  was  really  a  'part  2 
sea  duty  screen.’  Which  keeps  things  interesting  [sic].) 

I  am  not  suggesting  that  false  positives  are  common,  or  that  the  system's 
sensitivity  is  too  high,  but  ESSENCE  account  holders  believe  they  are.  They  are 
requesting  training,  in  particular,  how  to  detect  outbreaks  and  how  to  search  through  the 
alerts  and  false-positives  for  an  actual  alert. 

Third,  training  will  likely  mitigate  at  least  some  of  the  problems  that  have  been 
described.  Many  of  the  respondents  suggested  that  “hands  on  training  in  practical  terms 
for  your  particular  command/MTF”  is  preferred.  Although  ESSENCE  users  have  had 
initial  training  on  how  to  log  in  and  prepare  their  site  selections  to  begin  their 
investigations  and  to  use  the  alert  module  and  run  syndromic  and  reportable  diseases 
queries,  most  do  not  understand  what  the  time-series  tool  contributes  to  and  how  to 
properly  search  through  the  alerts  to  detect  an  outbreak.  Practical  lessons  should  be 
developed  to  help  educate  current  and  future  users. 
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B.  RECOMMENDATIONS 


To  mitigate  the  password  issues  one  recommendation  that  can  be  made  is  to 
modify  ESSENCE  to  allow  CAC  enabled  access.  This  would  eliminate  the  need  for 
users  to  have  to  reset  passwords  on  a  very  frequent  basis,  which  will  ultimately  increase 
the  value  and  desire  to  use  the  system.  This  approach  allows  the  frequently  deployed 
service  member  the  ability  to  gain  access  once  they  arrive  back  into  their  positions  to  log 
into  ESSENCE  without  reacquiring  an  account  and  wasting  up  to  a  month  before  they 
can  begin  using  ESSENCE. 

To  help  address  the  issue  of  miscodings,  perhaps  modify  ESSENCE  to  allow 
corrections  to  be  made  by  the  account  holder  who  has  access  to  CHCS  and  AHLTA, 
providers  who  verify  the  update,  and  by  the  laboratory  technicians.  If  the  users  have  the 
ability  to  change  the  miscodings  to  the  correct  codes,  then  perhaps,  this  may  help  bring 
down  the  alerts  rate  while  improving  ESSENCE  sensitivity  to  detect  actual  outbreaks. 
Another  possible  fix  for  miscodings  is  to  allow  the  individual  who  originally  input  the 
data  incorrectly  to  change  and  delete  the  old  entry.  Or  as  stated  in  the  survey,  which  96 
percent  of  the  respondents  agreed  with,  is  to  bold  the  finalized  ICD-9  code  and  syndrome 
that  will  allow  the  ESSENCE  monitors  to  focus  their  investigations  on  actual  PIN 
information  and  alerts. 

Lastly,  in  terms  of  training,  practical  applications  either  via  Web-based  video 
training  or  hands  on  training  in  the  workplace  will  substantially  increase  user 
understanding  in  the  system  and,  hopefully,  will  decrease  their  beliefs  that  false  positives 
are  the  only  thing  that  ESSENCE  “detects.”  Users  generally  believe  ESSENCE  is  a 
valuable  system,  and  with  the  proper  tools,  account  holders  will  be  able  to  detect 
communicable  diseases  earlier  in  their  area  and  provide  the  necessary  situational 
awareness  about  these  same  diseases  to  enhance  the  security  of  the  United  States  and 
throughout  the  world  with  healthy  members  of  the  military  and  their  families. 

C.  FUTURE  RESEARCH 

This  survey  was  only  administered  to  Navy  and  Marine  Corps  active  and  disabled 
account  holders,  but  I  recommend  that  future  research  focus  on  Air  Force  and  Army 
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account  holders  to  see  if  the  problems  areas  are  similar  and  if  not,  are  they  better  or 
worse.  If  they  are  better,  then  perhaps  joint  implementation  of  ESSENCE  could  be 
taught  to  the  different  military  services  to  continue  to  improve  the  system  and  allow  users 
to  value  it  more.  If  all  members  of  the  Armed  Forces  have  the  same  main  problem  areas, 
then  a  fix  in  the  system  and  thorough  training  can  be  provided  to  everyone. 
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APPENDIX  A.  USER  SURVEY 


Official  ESSENCE  User  Survey  (BUMED  6220-3) 


ABOUT  THIS  SURVEY 

Introduction.  You  are  incited  to  participate  in  a  survey  lo  help  the  Navy  anc  Marine  Corps  Public  Health  Center  {NMCPHC)  better  unde  stand 
hew  :he  Electronic  Surveillance  System  for  Early  Notification  of  Community-based  Epidemics  (ESSENCE)  is  being  implemented  and  used  in 
Ihe  Navy  and  Marine  Coras.  The  purpose  of  he  survey  Is  lo  provide  NMCPHC  with  in'ormalion  about: 

m  Whelhe"  and  how  users  have  been  tra  ned  lo  us#  ESSENCE: 

•  How  ESSENCE  is  currently  being  employed  by  tie  Navy  and  Marine  Corps:  and, 

*  How  effective  users  think  ESSENCE  Is  for  outbreak  detection  and  situational  awareness. 

Procedures,  Tnis  web-based  survey  is  being  conducted  by  the  Naval  Postgraduate  School  (NP3)  on  behalf  of  NMCPHC.  it  should  fake  no  more 
than  20*23  minutes  to  complete. 

Risks,  The  potential  risks  of  participating  in  this  study  ere  inadvertent  disclosure  of  individual  survey  responses.  To  mitigate  this  risk,  ihe  project 
has  implemented  extenshre  data  safeguarding  procedures:  a  copy  or  the  Data  Safeguarding  Plan  is  ayallaoie  from  the  survey  project  leader, 

LT  Randl  Korman.  Yet.  even  with  such  procedures  in  place,  there  always  remains  some  risk,  however  small,  of  a  data  breach. 

Ben&llts.  This  study  is  designed  to  provide  NMCPHC  with  recommendatiors  to  improve  ESSENCE  perfcmance,  training,  and  usability  of  the 
system.  This  is  your  oupcrlun  ty  lo  provide  NMCPj  1C  with  your  fru'h  and  her  out  feedback  so  decisions  car  be  made  lc-  most  benefit  current  and 

lulu  re  ESSENCE  users. 

Compensation,  No  tangiale  compensation  will  be  given,  it  you  would  like  to  receive  a  copy  of  tiie  report,  send  an  email  to  rmkorman@nps.edu 
with  the  word  'fAfCf.L/D£vj>r  She  subject  line  and  you  wifi  be  added  to  the  dislribulion  list.  The  lesulls  wil  be  available  In  the  March  2011 
timeframe  from  NMCPHC  via  Ms.  Asha  Riegodedios,  (904)  542-4o35,  asha  riegodedios2@med.navy.mil. 

Confidential  I  ly  &  Privacy  Act,  Any  personally-identifying  information  that  in  obtained  during  ths  study  will  be  kept  confident  al  to  the  full 
extent  permitted  by  law  and  your  participation  in  the  survey  will  net  be  disclosed  outside  of  the  NFS  team.  Vour  survey  responses  may  be 
merged  with  demographic  data  provided  by  the  NMCPHC  for  purposes  of  analysis.  All  efforts  will  be  mad#  by  NPS  to  keep  your  personally- 
identifying  information  confidential,  but  total  confidentiality  earn'd  be  guaranteed,  li  is  possible  that  NPS  is  requiied  to  divulge  Information 
obtained  in  the  course  of  this  research  to  the  subject's  chain  of  command  o- other  legal  body.  Lpon  completion  of  the  survey  the  team  will  de- 
identify  the  survey  results  to  ensure  complete  anonymity  and  the  de -id entitled  data  wll  be  provded  to  NMCPHC. 

Voluntary  Nature  ol  the  Study.  Participation  in  this  study  is  strictly  voluntary,  and  if  agreement  to  participation  is  given,  it  can  be  withdrawn  al 
any  time  without  prejudice. 

Poims  of  Contact,  Please  contact  Hne  surveyproiect  leader  and  co-investigator.  LT  Flandi  Kerman,  360 ‘929 -9343,  rimkor m an @ nos.edu.  with 
any  c-iustiu' s  uuoit : "ns  sj  vw.  The  principle  Investigator  is  Dr.  Fon  Frlcker,  63l-&56'304a,  rdfricke@nps.edu.  Questions  about  your  rights  as  a 
research  subject  or  any  other  concerns  may  tc  addressed  to  the  Navy  Postgraduate  School  IR0  Chair,  CAPT  John  Schmidt,  L'3N,  fl3l  -CDG* 
3B76,  ikschmld@nps.edu. 

Please  click  on  the  "YES"  button  to  indicate  that  you  voluntarily  agree  lo  take  the 
survey. 

r  yes  r  no 

In  thia  section  we  are  interested  in  how  you  are  currently  using  the  ESSENCE  3yatem. 
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Q2 


Q3 

Q4 

Q5 

Q6 

Q7 


On  average,  how  often  to  do  you  log  into  ESSENCE? 

C  Daily 

C  Weekly  (a  couple  lintel  per  week) 

C  Msnlhly  (a  coupla  tlmas  per  imoiUi) 

C  Greater  than  monthly  (a  couple  limes  par  year) 

C  Wfluar 

Do  you  receive  e-mail  alerts  from  ESSENCE? 

r  y#6 

r  no 

C  I  didn't  know  I  could  receive  a-rrail  aferls 

Do  you  log  Into  ESSENCE  only  If  you  receive  an  e-mail  alert? 

C  Yes 

r  no 

Which  e-mail  alert(s)  do  you  receive? 

C  Yellow  alerts 
C  Red  alerls 
C  Both  alerts 
C  I  don’t  <now 


Please  explain  why  you  do  not  receive  e-mail  alerts. 


Do  you  monitor  the  Alert  List  modulo? 

T  Yus 

r  no 

C  I  dnn’l  knnw  whal  this  ft 
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Q8 


Q9 


Q10 

Qll 

Q12 


Please  explain  why  you  do  not  use  the  Alert  List  to  monitor  alerts. 


In  this  spi’tion  we  are  interested  En  hnvu  you  run  a  reportnbe  disensn  Query. 

On  average,  how  often  do  you  run  a  reportable  diseases  Query? 

C  Daily 

C  Wfr&k  y  [a  eoupl*  times  per  w*ek| 

C  Monthly  (a  couple  times  per  month) 

C  Mere  lihan  monthly  (a  couple  times  per  year) 

C  Newer 

C  I  don't  know  what  this  Is 

Do  you  have  a  Navy  Disease  Reporting  System,  internet  (NDRSi)  account? 

C  Y <*; 

r  no 

If  confirmed,  do  you  or  do  you  with  a  co-worker  verify  that  the  reportable  diseases  you 

see  in  ESSENCE  are  also  inputed  into  NDRSi? 

r  y*k 
r  no 

how  much  do  you  agree  or  disagree  with  the  following  statement  ? 

I  lind  that  most  often  the  reportable  diseases  I  see  in  ESSENCE  are  rule-outs. 

C  Stioncly  Aprae 
C  Acres 
C  Neutral 
C  Dssag'ee 
C  Strongly  Disagree 
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Q13 


Q14 


Q15 

Q16 


Q17 


Please  explain  why  you  do  not  use  the  reportable  diseases  Query  module 


In  this  section  we  am  interested  in  hew  you  run  a  syndromic  Query  module. 

On  average,  how  often  do  you  run  a  syndromic  Query? 

C  Daily 

C  Weekly  'a  couple  iim&s  per  weak} 

C  Monlhly  (a  couple  tlnres  pen  month) 
r  More  than  monthly  (a  couple  timet  per  year; 

C  Never 

C  I  dorfl  knott  what  this  Is 

When  you  run  a  syndromic  Query,  how  often  do  you  use  the  following  data  sources? 


Always 

Frequently 

Sometimes 

Rarely 

Never 

SADR 

r 

r 

r 

r 

r 

Pharmacy 

r 

r 

r 

c 

r 

Laboratory 

c 

c 

r 

r 

r 

Fad  o  logy 

c 

r 

r 

r 

r 

When  you  run  syndromic  Query,  how  often  are  the  following  parameters  important  to 


your  investigation? 

Always 

Frequently 

Sometimes 

Rarely 

Never 

Age  Range 

r 

r 

r 

c 

r 

Clinic  Type 

r 

r 

r 

r 

r 

Starl  Date 

r 

r 

r 

C 

r 

End  Daie 

r 

r 

r 

c 

r 

Palisnt  Dalegory 

r 

r 

r 

r 

r 

Patient  Mil  iary  Branch 

r 

r 

r 

r 

r 

For  a  syndromic  Query,  do  you  do  the  following  during  your  investigation? 

Ye* 

I  narrow.  down  (or  opacity)  paHenl  paramotoie  depending  p 

on  tfrs  ayod  rome/djsftflss  I  im  mvsatlgat  ng 

I  loe<  at  syndromic  patterns  over  lime  C 

I  Investigate  certain  alerts  c  I  splayed  on  ihe  data  table  p 

Dy  clicking  on  the  Data  Details  for  a  given  day 
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Q18 


Q19 

Q20 

Q21 


Q22 


Please  explain  why  you  do  not  use  the  syndromic  Query  module. 


In  Ihis  section  w«  are  interested  in  learning  about  Lite  process  of  huw  you  investigate  an  alert  by  using 
either  the  Alert  List  or  the  Query  module. 

Do  you  investigate  alerts? 

C  Yes 
r  Wo 

Do  you  use  ESSENCE  to  investigate  alerts? 

r  Y*t 
C  Wo 

When  you  begin  an  investigation  using  ESSENCE,  which  of  the  following  site  selections 
do  you  choose? 

Yes  No 

Your  Ifica!  MTF  C  C 

Only  Navy  and  Mari  i  a  Corse  UTF'e  in  p  ^ 

yotr  riglen 

All  services' MTF's  n  y&urr&g  on  C  C 

Across  all  Navy  and  Marina  Corps  IUTFs  In  p  p 

yotr  c&un:ry 

Across  ell  sefiflces'  MTFs  in  your  country  C 

Once  you  notice  a  yellow  or  red  alert  for  a  particular  syndrome,  de  you  look  up  data 

details  within  ESSENCE  to  verify  the  need  for  additional  investigation. 

C  Yas 

c  wo 
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Q23 


Q24 


Q25 

Q26 


How  often  do  you  check  the  following  data  details  to  look  for  clustering  within  an  alert? 


Always 

Frequently 

Sometimes 

Rarely 

Never 

PIN 

r 

r 

c 

c 

r 

Encounter  D&la 

r 

r 

r 

r 

r 

Age 

r 

r 

r 

r 

r 

FMP 

r 

r 

r 

r 

r 

ICC'-S  Descriptions 

r 

r 

r 

r 

r 

Clinic  Type 

r 

c 

r 

r 

r 

Provider 

r 

r 

r 

r 

r 

Nl  IP 

r 

r 

r 

r 

r 

PatCal  (Patient  Cateoa-y) 

r 

r 

r 

r 

r 

Look  3A  the  tr  ap  view  (or  a 

r 

r 

r 

c 

r 

spatial  representation 

Otter  please  spadly) 

1  1 

How  often  do  you  use  the  following  external  tools  to  further  investigate  clustering  within 

an  alert? 

Always 

Frequently 

Sometimes 

Rarely 

Never 

AULT  A 

r 

r 

r 

r 

r 

CHSS 

r 

r 

r 

r 

r 

3peakwllh  providers 

r 

r 

r 

r 

r 

Speak  with  labcrratD'y 

r 

r 

r 

r 

r 

teemkiars 

Speak  with  Immunizations 

r 

r 

r 

r 

r 

tseinkiars 

fitter  -please  spardly) 

1  1 

Do  you  maintain  a  log  or  take  notes  {in  Word  or  Excel)  of  an  investigated  alert  for  future 

reference? 

C  Always 

C  Defends  cn  the  sy  ndroms/d  is &as#  investigated  and/o-r  the  severity  a-f  the  autbrefk  investigated 
C  klever 

After  researching  on  alert,  do  you  brief  your  command? 

C  Always 

C  Depends  cn  the  i y  ndremo/d  is  aas-u  in/estigaled  andVar  th«  severity  af  the  aulbrask  invastigatad 
C  Waver 
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Q27 


Q28 


Q29 

Q30 


Please  explain  why  you  do  not  investigate  alerts. 


If  you  do  not  use  ESSENCE  to  investigate  an  alert,  what  alternate  steps  do  you  take  to 


jin  :hls  sec-tlcpi  ws  ae  liiteie&tsd  in  understanding outbreak  detection  while  uslny  ESSENCE. 


Have  you  discovered  an  actual  outbreak  using  ESSENCE? 

r  Vea 
r  wo 

If  you  have  answered  yes  to  the  above  question,  check  all  the  outbreak  types  you  have 

discovered  using  ESSENCE. 

r  il. 

r~  Halt 

r  fii 

I-  Respiratory 

I-  Febrile 
I-  Neurological 
Rash 
\~  Nene 

Olher  (please  specify) 

I 

In  Ihis  section  wo  ore  interested  in  how  you  handle  miscodings. 
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Q31 


Q32 


Q33 


Have  you  observed  a  miscoded  ICD-9  code  in  ESSENCE? 

C  ’tes 
C  Wo 

What  steps  do  you  take  to  fix  miscodings? 

Check  all  that  apply. 

I  speak  with  the  provider 

J-  I  speak  wilK  the  inmumzalians  repress  n:ative 
J-  I  speak  wilh  the  MFT  coding  re  presents!  ve 

J-  I  speak  wilh  the  prevenlative  n edema  representalive  al  Ihe  clinic/hospita 
\~  I  do  no:  speak  wi:h  anyone 
Other  (please  spscily) 


In  this  section  we  are  Interested  in  your  opinions  of  the  ESSENCE  system. 

How  much  do  you  agree  or  disagree  with  the  following  statements  regarding  the 
ESSENCE  program? 


Sifnnflly 

Aijrftft 

Agree 

Neutral 

Disagree 

Strongly 

DIsagrAa 

I  real  oamforiaPi#  using  me  essence  program 

r 

r 

r 

r 

r 

ESSENCE  provides  my  department  nrlth  sltjalonal  awareness  aheot  communicable 

r 

r 

r 

r 

r 

diseases  of  Irterast  in  my  area 

ESSENCE  allows  me  tc  monitor  tie  '.rends  of  cisaasos  Jr  m y  area  ove'  line 

r 

r 

r 

r 

r 

ESSENCE  allows  me  to  1o  low  partlcjlai  reporlabla  diseases  in  my  area 

o 

r 

G 

c 

r 

ESSENCE  w-sulo  be  easle1  to  use  if  a  patient's  final  diagnosis  was  viewed  differently 

r 

r 

r 

r 

r 

In  tie  system  perhaps  If  bolded 

ESSENCE  woulc  be  easier  lo  use  if  a  patient's  miscoded  1  CD-9  code  were  updaled 

r 

r 

c 

r 

r 

and  correctly  inputted  into  Ihe  system 

The  Urn  o -comae  module  in  ESSENCE  ie  useful  la  identify  oulbroalse 

r 

r 

c 

r 

r 

1  find  thal  the  alerts  in  ESSENCE  are  often  false -positives 

r 

r 

r 

r 

r 

In  Ihis  section  we  are  interested  in  learning  about  your  experience  n  obtaining  your  current  ESSENCE 

account 
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Q34 


Q35 

Q36 

Q37 


How  long  did  it  take  you  to  acquire  your  current  ESSENCE  account? 

C  1  ■  IS  days 
C  1 6  ■  30  days 
C  31  ■  45  days 
C  46  ■  60  days 
C  Wore  than  60  days 
C  I  don'l  'e-member 

Did  you  have  to  apply  more  than  once  to  obtain  your  current  ESSENCE  account? 

r  Yos 

C  Wo 

Did  you  have  to  speak  with  an  ESSENCE  helpdesk  representative  while  waiting  to 
access  your  current  ESSENCE  account?  (Including  e-mails  and  phone  calis) 

if*  Yfts 

C  Wo 

C  I  don'l  ■£  member 

Estimate  how  many  times  you  needed  to  contact  the  helpdesk  (via  e-mail  or  phone  call] 
to  acquire  your  current  ESSENCE  account. 

C  1  lime 
C  2  limes 
C  3  times 
C  4  times 
C  3  or  more  :lm*s 
C  I  don'l  'emember 

In  this  section  we  are  interested  in  learning  about  what  training  you  have  had  on  the  ESSENCE  system 
and  what  you  would  like  to  see  in  the  future. 
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Q38 


Q39 


Q40 


Who  did  you  receive  your  ESSENCE  training  from? 
Check  all  that  apply. 

J-  My  pr-frd^es&or 
I-  My  toss 
|  I  am  sell-LaugtiE 

|  I  dawn  loaded  Ihe  ESSENCE  training  modules  and  tulerali 

I  awarded  a  ^odertnci  where  I  was  taught  hew  to  work  :he  ayatenn 
I-  I  waa  taught  by  :ho  NE3MU 
Olhar  (ploace  specify) 


Where  would  you  like  to  see  training  perlormed  lor  current  and  Future  users  of  the 
ESSENCE  system? 


Yes 

Nc 

At  my  own  d&sk 

C 

r 

In.  public  health  conlerances 

C 

r 

In  h-ospltal/elfnte  meetings 

c 

r 

ln"C  sen  eel 

filhfli  fplMSfl  sinr.ily) 


How  would  you  like  to  see  training  provided  for  current  and  future  users  of  the 
ESSENCE  system? 


Audit  and  videu  based 
Pcwerpoint  ptaaerv^tlena 
Handi'On  training 

GiMsr  fpluitta  tsuBdly) 


Yes 

c 

r 

r 


r 

r 

r 
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Q41 


Q42 


Q43 


Q44 


What  type  of  training  would  you  like  for  current  and  future  users  of  the  ESSENCE 
system 


InrtiaE  Ira  mine  to  uae  ESSENCE 
Site  selection  se:  up 

H(w  tc  tivalu-jla  dials 

Query  module  applications 
OuibrutiK  scnnarfcs 
Other  iplaase  spscjferi 


Ves 

r 

r 

r 


Do  you  have  any  suggestions  that  wit!  help  make  the  ESSENCE  system  easier  to  use? 


Do  you  have  any  suggestions  that  will  help  make  the  ESSENCE  system  more  effective? 


Do  you  have  any  other  comments  that  you  would  like  to  make  regarding  the  ESSENCE 
system? 
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Q45 


Q46 


Q47 


Q48 


Select  the  current  user  level  that  best  describes  you 

C  Single  MTT  level  user 
C  Multiple  MTF‘s  level  user 
C  NEPMU  level  user 
Other  (please  specify) 

I  I 

Select  lire  facility  that  you  currently  support. 

C  Clinic 
r  Naval  Hospital 
C  Naval  Medical  Career 
C  NEPMU 
Olhar  ip  ease  specify) 

I  I 

Check  all  the  service(s)  you  currently  support. 

|~  U.S.  Air  Farce 
U.S.  Army 

f~  U.S.  Marine  rjorps 
f~  U.S.  Navy 

Select  the  professional  background  that  best  describes  you. 

C  Corpsman 

C  Preven tative  Msdmin e  Tech niclan 
C  Epidemic  legist 
C  Enwir&nmrental  Health  Officer 
r  Preventive  Medicine  Officers 
r  Nnse 
C  Physician 
Olher  (please  specify) 
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Q49 


Which  category  do  you  fall  into? 

C  El  •  E3 
C  E4  -  E6 
C  E? .  ES 

r  0103 
r  o^-o& 

r  Civilian 
r  Contractor 

II  you  are  a  civilian  cr  a  contactor,  pitas  &  enter  your  grade  level  orequivalen!  (GS-11,  WG-9,  WL-11,  etc) 


Thank  ynn  fhr  rnmplfiting  this  survey  nn  hehalf  rtx  the  Naval  Pnslgrarlijate  Srhrnl  anrl  the  Navy  anri  Marine  Corps 
Publb  Health  Center. 
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APPENDIX  B.  IRB  DOCUMENTS 


Naval  Kostgraduatc  School 

[rrslPluii^nul  Krvicw  BuunJ  (IRB) 


F^ortr  PrvALd^nc.  F'vugntclynrc  Sirhwl 

Via:  CJmlrmnn.  IngHiHjEioriak  Ktvisw  Dawd  KT  I  ?  ^g| 

To:  DFi  Ronald  FVichot,  Jr.  Upcraiigos  Rwwch  Departmeni 

1  .T  Hnmh  Kfirman,  t  ISN 


SLT&J;  ASSESS!  NO  THE  BSSENCrDtOSU ft VEIiLAMCE  SYSTEM  AS  USED  BY 
THE  NAVY  AND  MARINE  CORPS:  USER  1  RAINING,  SYSTEM 
EMPLOYMENT  AND  PERCEIVED  VALUE 


Eik  I  ( I }  Apprcved  I RU  Prouxoi: 

1.  'Hie  IMPS  IRB  Is  pleased  lu  inform  y&u  [hat  fine  MRS  President  few s  approved  your 

project  (NFS  JRBff  NPS.J&1 1 .000 1  -JR-EP7«A)l  The  approved  IRB  Protocol  is  found  in 
mdosgre  ( l),  CumpJelion  o(  the  C]TJ  Rejeareh,  Elbics ']  has  been  cor.fi nrytd 

2.  This  approval  experts  cm  ^1  March  I  I .  ]f  additional  Lime  is  :equLred  to  complete  the 
je^ardi,  a  MMinulllg  review  report  musl  be  approved  by  the  IRB  arid  NFS  Pi*snfem 

prior  l-D  Lhe  expiration  of  approval.  Al  c^pirflUon  pJI  PCSCflrch.  (suhjjecl  KCnutotHK.  JlHA 
tcdkeiion,  analysis  of  data  cunlru nin^.  Pill  mU^-l  ctfi.Se 

3.  You  are  required  to  report  to  ihe  IRB  Bay  ucianlk i paicd  problems  at  serkju*  adverse 
evepeg  to  tluc  NPS  JftB  within  24  hours  Of  the  Ottiafttnee. 


J.  Any  pioposed  changes  in  IRB  approved  reMrch  He  reviewed  an d  tppvtmcd  fey 
ihc  NFS  JKB  and  NFS  PncsidenC  prior  to  ItnplenWfUUIIcm  CSCepi  where  neCCiSfry  to 
itrl  ij7i-i  iiiiiv  eppfCTflt  i  mined  iaie  bayards  lo  research  parttc-ipanls  and  adgccti 

J  As  lhfl  Principal  Investigator  il  is  your  responsibiLity  to  ensure  that  Ibe  research  and 
I  he  iJ^li^ns  uT  M-l!  pnjjecl  p^j^MHhbi:l  jnvuivcd  iii  cc-jadacLuig.  Ihi.3  aludy  wLIL  conform  w.th 
iht  ]RB  approved  protocol  rin.J  IRB  jequiremen'Si'pDlicies. 


6.  Afi«r  the  experiment  is  completed  the  Principe]  investigate!  will  .uibm](  iq  ihe  Hwmtt 
Subjects  Pnocrdion  Office,  all  signed  informed  c&nsenrt  idocuments*  un&nticipaird 
problem  reports.  iepOrt;>-  end  a  E^d  of  Efcperimem  Report.  The  Human 

Subject 5  Restflich  Office  will  Secure  these  documents  for  I  &  ye  an:  and  Then  forward  to 
the  p^aresl  J-KU. 

QjrtFt  Jolin  Sth.nkH,  M5C.  U5N 
Chair 

Irsiiruboiwl  Review  Board 


Dnjii^l  T.  Oliver 
President 

Naval  PoiigfadLLile  SefewJ 
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DEPARTMENT  (IF THE  NAVY 

NAVY  ANU  MARINI  ajRP*  ft  miC  HfeA4.H1  TbUTIR 

UPJOHN  paul jqwesciftCLt untf  i  m 
ftsosMOimi  v.a  sim-mi 


5300 
3er  Ph/ 


Fcomj  Commanding  Officer#  Navy  and  Marine  Corps  Fu&lic  Health 
Center 

To:  Superintendent,  Naval  Postgraduate  School 

Subj:  SUPFQRT  QF  S TO DEBT  SURVEY  PROJECT 

Reft  fa)  BUHEDINST  6220. 12B 

(b)  OPM AV INST  S 300. SC 

Enel:  (1)  EES  ESC  EL  User  Survey  Package 

1.  This  letter  is  to  authorize  NHCPHC  support  for  a  Naval  Postgraduate 
School  (NFS)  student  project  that  will  also  benefit  this  COfltmAndL  Per 
reference  £a),  the  Navy  and  Karine  Corps  Public  Health  Canter  is  tasked 
With  management  of  medical  surveillance  functions  AS  well  as  evaluation  pf 
program  areas  for  procsss  improvement .  To  support  this  effort,  one  of 
your  students  is  into res ted  in  Administering  a  survey  to  all  Navy  Medicine 
Electronic  Surveillance  System  for  the  Early  Mo ti float ion  of  Conmunity- 
based  Epi demies  {ESSENCE)  users  as  part  of  her  thesis  project.  LT  Rand! 
Kerman  has  been  working  with  members  of  my  Preventive  Medicine  Department 
in  the  design  of  this  survey  and  has  created  the  draft  package  (enclosure 
(1)),  The  next  step  is  for  LT  Kerman  to  seek  approval  from  the  NFS 
Institutional  Review  Board  as  well  Wavy  Survey  approval  per  reference  £bl . 

2.  Once  the  IR.B  and  survey  approvals  are  received,  NHCPHC  will  provide 
NPE  the  email  addresses  of  Navy  ESSENCE  users  as  well  AS  A  pie-survey 
small  notice  to  those  users  to  facilitate  IT  Kontian's  survey  project.  It 
is  anticipated  that  LT  KO£manr£  results  can  be  used  for  improving  this 
important  medical  surveillance  program  so  we  are  pleased  to  assist  in  this 
joint  venture. 


3.  Point  of  contact  is  CDR  Christopher  Clagefct,  DSN  377-0715  or  {757) 
9  53 - 07 1 5  or  ema i 1  at  chi i s  top  her . 01 A get  timed , navy ,  mi  1 . 


3 .  jft  .  COHEN 


Cdpy  td : 
BUMED  M3/5 
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Data  Safeguarding  Plan 


Associate  Professor  R.D.  Fricker,  Jr.,  Principal  Investigator 
LT  Randi  Korman,  Co-investigator 

August  13,  2010 

Project  Description 

The  research  consists  of  a  survey  of  ESSENCE  monitors  and  subsequent  analysis 
of  the  resulting  data.  The  survey  will  be  conducted  as  a  thesis  project  on  behalf  of  the 
Navy  and  Marine  Corps  Public  Health  Center  (NMCPHC).  The  instrument  will  consist 
of  questions  about  how  extensive  the  user  training  was  prior  to  program  operation, 
system  employment,  and  their  perceived  value. 

Description  of  Data  to  be  Acquired  and  their  Usage 

Survey  data  will  be  acquired  from  current  users  of  the  ESSENCE  biosurveillance 
system  via  a  Web-based  survey  instrument.  If  possible,  this  data  will  be  merged  with 
respondent  demographic  information  obtained  from  the  NMCPHC  that  will  include  last 
name,  first  name,  e-mail,  and  Medical  Treatment  Facility  (MTF)  or  Navy  Environmental 
Preventive  Medicine  Unit  (NEPMU).  This  information  will  be  used  to  personalize  the 
survey  e-mail  invitations  and  for  analysis  of  results  by  the  team  members. 

Data  Sensitivity 

The  personal  identifying  information  (PII)  that  will  be  acquired  from  the 
NMCPHC  for  use  in  this  survey  will  be  the  ESSENCE  monitor’s  e-mail  addresses,  last 
name,  first  name,  and  MTF  or  NEPMU  affiliation.  Some  demographic  information  may 
be  sensitive  to  some  individuals.  However,  the  survey  data  itself  will  consist  of 
individual  responses  to  questions  about  user  training,  system  employment,  and  perceived 
value,  a  fairly  innocuous  subject  that  poses  minimal  risk  to  survey  respondents. 
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Data  Safeguarding  Responsibilities 

The  Principal  Investigator  (PI)  and  Co-investigator  bear  sole  and  complete 
responsibility  for  safeguarding  the  data.  Only  the  PI  and  Co-investigator  will  have  access 
to  the  complete  data  set.  They  will  ensure  that  all  provisions  of  this  Data  Safeguarding 
Plan,  as  well  as  any  other  requirements  levied  by  the  NMCHPC,  are  fully  and  completely 
implemented. 

Data  Safeguarding  Procedures 

The  PI  and  Co-investigator  will  implement  the  following  procedures  for 
safeguarding  the  data  on  this  project: 

1 .  All  requirements  for  safeguarding  personally  identifiable  information  (PII) 
outlined  in  NAVPGSCOL  Instruction  2201  dated  5  July  2007  will  be  fully 
implemented  and  followed. 

2.  The  data  will  be  collected  using  a  commercial  Web  survey  program 
(SurveyMonkey).  All  survey  data  will  be  collected  using  enhanced  SSL 
encryption  to  protect  respondent  information  during  transmission.  Once 
data  collection  is  complete,  the  data  will  be  downloaded  to  a  file  system  at 
NPS  and  the  data  deleted  from  the  SurveyMonkey  server. 

3.  Data  files,  lists  or  any  other  reports  or  printouts  that  link  personnel  names 
or  contain  personal  or  other  sensitive  data  will  never  be  received  or 
transmitted  via  e-mail.  The  preferred  method  of  receiving  such 
information  is  via  a  common  shared  drive  on  the  NPS  intranet.  In 
accordance  with  NAVPGSCOL  Instruction  2201  dated  5  July  2007,  the 
project  will  only  transport  PII  data  “using  the  secure  server  and  following 
the  NPS  encryption  guidelines.” 

4.  Respondent  contact  information  will  be  strictly  limited  to  members  of  the 
survey  team  on  a  need  to  know  basis. 

5.  All  such  data  will  only  be  stored  on  computers  and  file  systems  physically 
located  within,  and  will  not  be  removed  from,  the  physical  confines  of  the 
Naval  Postgraduate  School.  In  addition,  data  files  containing  personal  or 
other  sensitive  data  will  never  be  stored  on  thumb  drives  or  laptop 
computers. 

6.  Working  files  of  the  original  data  files  will  only  be  stored  on  a  password 
protected  network  drive  physically  located  within  the  confines  of  NPS. 
Any  other  media  (e.g.,  CD-ROM)  containing  data  that  is  received  or 
created  will  also  be  stored  in  a  secure  locker  in  the  System  Technology 
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Battle  Laboratory  (STBL)  for  as  long  as  it  is  needed  by  the  project  and 
then  destroyed.  The  STBL  is  supervised  during  normal  working  hours  and 
kept  locked  after  hours  at  all  times. 

7.  Sensitive  hard  copy  lists  and  reports  will  be  stored  in  a  secure  locker  in  the 
STBL  when  they  are  not  being  used.  Access  to  these  materials  will  be 
limited  to  the  PI  and  Co-investigator. 

8.  Identifiable  information  will  not  be  printed  unless  absolutely  necessary. 
Such  printing  should  be  very  rare.  Printed  output  containing  identifiers 
will  be  treated  as  confidential.  Printouts  containing  identifiable  data  will 
be  destroyed  by  shredding  when  no  longer  needed. 

9.  Analytical  files,  constructed  from  the  original  data  files,  will  not  contain 
any  personal  data,  including  names,  SSNs,  or  other  information  from 
which  an  individual  can  be  identified.  This  de-identified  analytical  file 
will  contain  an  identifying  number  that  uniquely  links  each  individual  in 
the  analytical  file  to  the  relevant  records  in  the  original  data  files. 

10.  The  cross-walk  file  from  the  identifying  information  to  unique  identifying 
number  will  be  secured  in  the  same  manner  as  the  original  files  (see  item  3 
above). 

1 1 .  The  de-identified  analytical  file  or  files  will  only  be  stored  on  a  password 
protected  internal  (“H”)  drive.  . 

12.  The  project  will  maintain  a  log  of  all  sensitive  computer  files  and  all 
media,  output  and  reports  containing  sensitive  data  contained  in  the  secure 
locker.  Directories  containing  sensitive  files  will  be  monitored  on  a 
regular  basis  to  ensure  that  file  permissions  are  correctly  set  on  all  files. 

13.  Upon  completion  of  the  project,  all  electronic  files  with  identifiable 
information  will  be  positively  erased  using  a  utility  that  overwrites  the 
files  on  the  hard  drive,  all  other  data  storage  media  will  be  physically 
destroyed  in  such  a  manner  that  data  recovery  is  impossible  (i.e., 
shredding  CD-ROM  disks),  and  all  paper  files  with  personal  or  other 
identifying  information  will  be  destroyed  by  shredding. 

14.  Any  serious  violation  of  the  Data  Safeguarding  Plan  will  be  reported  in 
writing  to  the  NPS  Institutional  Review  Board,  with  a  copy  to  the  NPS 
Operations  Research  Department  Chair. 
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Audit  and  Monitoring  Plans 

This  Data  Safeguarding  Plan  will  be  reviewed  periodically  to  ensure  that  it 
continues  to  meet  the  needs  of  the  project,  as  well  as  that  the  project  is  following  all 
procedures  and  requirements  outlined  herein.  Logs  will  be  reviewed  as  necessary  to 
ensure  they  are  up-to-date.  A  periodic  review  of  file  permissions  will  be  conducted  to 
ensure  that  file  permissions  are  correctly  set  and  maintained  on  all  files. 
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REPORT  ANALYSIS  DATA 
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APPENDIX  C.  E-MAIL  NOTIFICATIONS 


(Pre-Notification  E-mail): 

From:  christopher.clagett@med.navy.mil 
To:  [E-mail] 

Subject:  ESSENCE  User  Survey 
Dear  [FirstName]  [LastName], 

Tomorrow  you  will  receive  an  e-mail  invitation  to  participate  in  a  brief  survey  being  conducted  by  the 
Naval  Postgraduate  School  (NPS)  about  the  Electronic  Surveillance  System  for  Early  Notification  of 
Community-based  Epidemics  (ESSENCE)  system.  This  survey  was  developed  by  NPS  on  behalf  of  the 
Navy  and  Marine  Corps  Public  Health  Center  (NMCPHC). 

The  purpose  of  the  survey  is  to  provide  NMCPHC  with  information  about: 

•  Whether  and  how  users  have  been  trained  to  use  ESSENCE; 

•  How  ESSENCE  is  currently  being  employed  by  the  Navy  and  Marine  Corps;  and, 

•  How  effective  users  think  ESSENCE  is  for  outbreak  detection  and  situational  awareness 
In  short,  NMCPHC  wants  to  better  understand  how  to  better  facilitate  ESSENCE  use  in  the  Navy. 

The  purpose  of  this  e-mail  is  to  provide  you  with  advanced  notice  and  request  your  participation.  You  will 
receive  an  e-mail  tomorrow  from  LT  Randi  Korman  (rmkorman@nps.edu)  with  the  subject  line:  ESSENCE 
Monitor  Survey.  Please  click  on  the  link  that  will  be  provided  as  the  survey  is  entirely  online. 

We  look  forward  to  your  participation  and  thank  you  in  advance. 

Sincerely, 

Christopher  D.  Clagett,  MD,  MPH,  MSBA 
CDR,  MC,  USN  (FMF) 

Head,  Preventive  Medicine  Dept 

Navy  &  Marine  Corps  Public  Health  Center 

Please  wait  until  tomorrow  to  receive  the  official  survey. 

[Survey  Link] 

This  link  is  uniquely  tied  to  this  survey  and  your  e-mail  address.  Please  do  not  forward  this  message. 

Please  note:  If  you  do  not  wish  to  receive  further  e-mails  from  us,  please  click  the  link  below,  and  you  will 
be  automatically  removed  from  our  mailing  list. 

[RemoveLink] 
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(Notification  E-mail): 

From:  rmkorman@nps.edu 
To:  [E-mail] 

Subject:  Official  ESSENCE  User  Survey  (BUMED  6220-3) 

Dear  [FirstName]  [LastName], 

As  mentioned  in  CAPT  Clagetf  s  e-mail  yesterday,  you  are  invited  to  take  a  survey  about  your  use  of  the 
ESSENCE  system.  The  purpose  of  the  survey  is  to  solicit  ESSENCE  users’  opinions  about  training, 
system  employment,  and  users’  perceived  value  of  the  system.  This  is  your  opportunity  to  provide  the 
Navy  and  Marine  Corps  Public  Health  Center  (NMCPHC)  with  feedback  about  what  is  working  and  what 
needs  improvement  in  the  ESSENCE  program. 

The  survey  is  now  available  via  the  following  link:  [SurveyLink] 

The  survey  takes  approximately  20-25  minutes  to  complete.  Note  the  above  link  is  uniquely  tied  to  the 
survey  and  your  e-mail  address.  Please  do  not  forward  this  message.  If  you’ve  received  this  message  in 
duplicate  and  have  already  completed  the  survey  please  disregard.  Also  note  that  all  information  collected 
in  this  survey  will  be  kept  strictly  confidential  and  will  only  be  accessible  by  the  Naval  Postgraduate 
School  (NPS)  investigators.  Only  anonymous  survey  results  will  be  forwarded  to  NMCPHC,  without 
identifying  information  (including  e-mail  address).  Any  information  identifying  your  participation  in  this 
survey  will  be  destroyed  by  NPS  upon  completion  of  the  analysis. 

Please  feel  free  to  contact  me  if  you  have  any  questions  regarding  the  survey.  Mrs.  Asha  Riegodedios 
(asha.riegodedios2@med.navy.mil;  COMM:  904-542-4635;  DSN:  942-4635)  is  your  NMCPHC  point  of 
contact  and  will  distribute  the  results  in  the  MAR  2011  timeframe. 

Thank  you  for  your  participation. 

Sincerely, 

LT  Randi  Korman 
rmkorman@nps .  edu 

xxx-xxx-xxx 

Please  note:  If  you  have  never  been  an  ESSENCE  user,  please  click  the  link  below.  We  apologize  for  any 
inconvenience. 

[RemoveLink] 
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(1st  Follow-Up  E-mail): 

From:  rmkorman@nps.edu 
To:  [E-mail] 

Subject:  Official  ESSENCE  User  Survey  (BUMED  6220-3) 

Dear  [FirstName]  [LastName], 

On  Monday,  November  8,  2010  you  received  an  e-mail  invitation  to  take  a  survey  about  your  use  of  the 
ESSENCE  system.  To  date  we  have  not  received  your  response.  The  purpose  of  the  survey  is  to  solicit 
ESSENCE  users’  opinions  about  training,  system  employment,  and  users’  perceived  value  of  the  system. 
This  is  your  opportunity  to  provide  the  Navy  and  Marine  Corps  Public  Health  Center  (NMCPHC)  with 
feedback  about  what  is  working  and  what  needs  improvement  in  the  ESSENCE  program. 

The  survey  is  now  available  via  the  following  link:  http:  [SurveyLink] 

The  survey  takes  approximately  20-25  minutes  to  complete.  Note  the  above  link  is  uniquely  tied  to  the 
survey  and  your  e-mail  address.  Please  do  not  forward  this  message.  If  you’ve  received  this  message  in 
duplicate  and  have  already  completed  the  survey  please  disregard.  Also  note  that  all  information  collected 
in  this  survey  will  be  kept  strictly  confidential  and  will  only  be  accessible  by  the  Naval  Postgraduate 
School  (NPS)  investigators.  Only  anonymous  survey  results  will  be  forwarded  to  NMCPHC,  without 
identifying  information  (including  e-mail  address).  Any  information  identifying  your  participation  in  this 
survey  will  be  destroyed  by  NPS  upon  completion  of  the  analysis. 

Please  feel  free  to  contact  me  if  you  have  any  questions  regarding  the  survey.  Mrs.  Asha  Riegodedios 
(asha.riegodedios2@med.navy.mil;  COMM:  904-542-4635;  DSN:  942-4635)  is  your  NMCPHC  point  of 
contact  and  will  distribute  the  results  in  the  MAR  2011  timeframe. 

Thank  you  for  your  participation. 

Sincerely, 

LT  Randi  Korman 
rmkorman@nps .  edu 

Please  note:  If  you  have  never  been  an  ESSENCE  user,  please  click  the  link  below.  We  apologize  for  any 
inconvenience. 

[RemoveLink] 
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(2nd  Follow-Up  E-mail): 

From:  rmkorman@nps.edu 
To:  [E-mail] 

Subject:  Official  ESSENCE  User  Survey  (BUMED  6220-3) 

Dear  [FirstName]  [LastName], 

Last  week  you  received  an  e-mail  invitation  to  take  a  survey  about  your  use  of  the  ESSENCE  system.  To 
date  we  have  not  received  your  response.  The  purpose  of  the  survey  is  to  solicit  ESSENCE  users’  opinions 
about  training,  system  employment,  and  users’  perceived  value  of  the  system.  This  is  your  opportunity  to 
provide  the  Navy  and  Marine  Corps  Public  Health  Center  (NMCPHC)  with  feedback  about  what  is 
working  and  what  needs  improvement  in  the  ESSENCE  program. 

The  survey  is  now  available  via  the  following  link:  http:  [SurveyLink] 

The  survey  takes  approximately  20-25  minutes  to  complete.  Note  the  above  link  is  uniquely  tied  to  the 
survey  and  your  e-mail  address.  Please  do  not  forward  this  message.  If  you’ve  received  this  message  in 
duplicate  and  have  already  completed  the  survey  please  disregard.  Also  note  that  all  information  collected 
in  this  survey  will  be  kept  strictly  confidential  and  will  only  be  accessible  by  the  Naval  Postgraduate 
School  (NPS)  investigators.  Only  anonymous  survey  results  will  be  forwarded  to  NMCPHC,  without 
identifying  information  (including  e-mail  address).  Any  information  identifying  your  participation  in  this 
survey  will  be  destroyed  by  NPS  upon  completion  of  the  analysis. 

Please  feel  free  to  contact  me  if  you  have  any  questions  regarding  the  survey.  Mrs.  Asha  Riegodedios 
(asha.riegodedios2@med.navy.mil;  COMM:  904-542-4635;  DSN:  942-4635)  is  your  NMCPHC  point  of 
contact  and  will  distribute  the  results  in  the  MAR  2011  timeframe. 

Thank  you  for  your  participation. 

Sincerely, 

LT  Randi  Korman 
rmkorman@nps .  edu 

Please  note:  If  you  have  never  been  an  ESSENCE  user,  please  click  the  link  below.  We  apologize  for  any 
inconvenience. 

[RemoveLink] 
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(3rd  Follow-Up  E-mail): 


From:  rmkorman@nps.edu 
To:  [E-mail] 

Subject:  Official  ESSENCE  User  Survey  (BUMED  6220-3) 

Dear  [FirstName]  [LastName], 

Two  weeks  ago  you  received  an  e-mail  invitation  to  take  a  survey  about  your  use  of  the  ESSENCE  system. 
To  date  we  have  not  received  your  response.  The  purpose  of  the  survey  is  to  solicit  ESSENCE  users’ 
opinions  about  training,  system  employment,  and  users’  perceived  value  of  the  system.  This  is  your 
opportunity  to  provide  the  Navy  and  Marine  Corps  Public  Health  Center  (NMCPHC)  with  feedback  about 
what  is  working  and  what  needs  improvement  in  the  ESSENCE  program. 

The  survey  is  now  available  via  the  following  link:  http:  [SurveyLink] 

The  survey  takes  approximately  20-25  minutes  to  complete.  Note  the  above  link  is  uniquely  tied  to  the 
survey  and  your  e-mail  address.  Please  do  not  forward  this  message.  If  you’ve  received  this  message  in 
duplicate  and  have  already  completed  the  survey  please  disregard.  Also  note  that  all  information  collected 
in  this  survey  will  be  kept  strictly  confidential  and  will  only  be  accessible  by  the  Naval  Postgraduate 
School  (NPS)  investigators.  Only  anonymous  survey  results  will  be  forwarded  to  NMCPHC,  without 
identifying  information  (including  e-mail  address).  Any  information  identifying  your  participation  in  this 
survey  will  be  destroyed  by  NPS  upon  completion  of  the  analysis. 

Please  feel  free  to  contact  me  if  you  have  any  questions  regarding  the  survey.  Mrs.  Asha  Riegodedios 
(asha.riegodedios2@med.navy.mil;  COMM:  904-542-4635;  DSN:  942-4635)  is  your  NMCPHC  point  of 
contact  and  will  distribute  the  results  in  the  MAR  2011  timeframe. 

Thank  you  for  your  participation. 

Sincerely, 

LT  Randi  Korman 
rmkorman@nps .  edu 

Please  note:  If  you  have  never  been  an  ESSENCE  user,  please  click  the  link  below.  We  apologize  for  any 
inconvenience. 

[RemoveLink] 
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(4th  Follow-Up  E-mail): 


From:  rmkorman@nps.edu 
To:  [E-mail] 

Subject:  Official  ESSENCE  User  Survey  -  A  Few  Comments/Results 
Dear  [FirstName]  [LastName], 

Early  this  month  you  received  an  e-mail  invitation  to  take  a  survey  about  your  use  of  the  ESSENCE 
system.  To  date  we  have  not  received  your  response. 

However,  we  have  received  response  from  124  monitors  so  far.  Below  are  some  of  their  opinions  about  the 
ESSENCE  system. 

Positive: 

•  “It  has  been  a  great  additional  tool  to  capture  reportable  diseases  and  track  for  outbreak  trending. 
Hope  it  is  a  program  that  Navy  Medicine  and  the  Tri-Services  keep  as  our  Preventive  Medicine 
Departments  keep  getting  smaller  with  continued  deployment  support  missions.” 

•  “Overall,  I  am  glad  that  ESSENCE  exists.  The  ability  to  perform  active  surveillance  is  wonderful, 
as  providers,  historically,  do  a  horrible  job  reporting  infectious  diseases  to  the  PMA.” 

•  “Great  system.” 


Negative: 

•  “Direct  access  patient  information  to  AHLTA/CHCS  and  NDRSi.  Too  much  time  is  spent 
switching  between  programs  to  find  patient  information  in  order  to  ensure  treatment  and  that  reporting  was 
completed.” 

•  “Get  rid  of  it.  Have  a  program  that  is  used  by  most  of  the  medical  community  (CHCS  or  AHLTA) 
do  disease  reporting.  When  we  do  reports  here  the  information  we  need  we  have  to  get  from 
AHLTA/CHCS  so  why  not  just  get  rid  of  the  middle  man  and  have  those  programs  report  the  same  things 
as  ESSENCE.” 

Your  response  is  important.  We  want  to  ensure  that  all  opinions  and  viewpoints  are  reflected  in  the  results. 
Please  click  on  the  following  link  to  take  the  survey:  http://www.surveymonkey.eom/s.aspx  .  It  will  take 
approximately  20-25  minutes  to  complete. 

Note  the  above  link  is  uniquely  tied  to  the  survey  and  your  e-mail  address.  Please  do  not  forward  this 
message.  Also  note  that  all  information  collected  in  this  survey  will  be  kept  strictly  confidential  and  will 
only  be  accessible  by  the  Naval  Postgraduate  School  (NPS)  investigators.  Only  anonymous  survey  results 
will  be  forwarded  to  NMCPHC,  without  identifying  information  (including  e-mail  address).  Any 
information  identifying  your  participation  in  this  survey  will  be  destroyed  by  NPS  upon  completion  of  the 
analysis. 

Please  feel  free  to  contact  me  if  you  have  any  questions  regarding  the  survey.  Mrs.  Asha  Riegodedios 
(asha.riegodedios2@med.navy.mil;  COMM:  904-542-4635;  DSN:  942-4635)  is  your  NMCPHC  point  of 
contact  and  will  distribute  the  results  in  the  MAR  2011  timeframe. 

Thank  you  for  your  participation. 

Sincerely, 

LT  Randi  Korman 
rmkorman@nps .  edu 

Please  note:  If  you  have  never  been  an  ESSENCE  user,  please  click  the  link  below.  We  apologize  for  any 
inconvenience.  [RemoveLink] 
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(5th  Follow-Up  E-mail): 

From:  christopher.clagett@med.navy.mil 
To:  [E-mail] 

Subject:  Request  for  your  Participation  in  the  ESSENCE  User  Survey 
Dear  [FirstName]  [LastName], 

Over  the  last  few  weeks  you  should  have  received  a  number  of  e-mail  invitations  from  LT  Randi  Korman 
to  participate  in  a  brief  survey  about  the  ESSENCE  system.  To  date  your  response  has  not  been  received. 

The  survey  was  developed  to  gather  information  about  ESSENCE  use,  training,  and  effectiveness.  One  of 
our  primary  purposes  in  soliciting  your  feedback  is  to  help  us  ascertain  the  information  needs  of  ESSENCE 
users,  as  well  as  the  effectiveness  of  training  we  have  made  available  to  date.  Your  input  is  very  important 
to  the  Navy  and  Marine  Corps  Public  Health  Center  so  we  can  better  understand  how  to  improve 
ESSENCE  use  in  the  Navy. 

Please  click  on  the  following  link  to  participate:  http://www.surveymonkey.eom/s.aspx 
Thank  you  for  taking  the  time  to  complete  the  survey. 

Sincerely, 

Christopher  D.  Clagett,  MD,  MPH,  MSBA 
CAPT,  MC,  USN  (FMF) 

Director,  Preventive  Medicine  Dept 
Navy  &  Marine  Corps  Public  Health  Center 

Note:  If  you  do  not  wish  to  participate  in  this  survey,  please  click  on  the  following  link:  [RemoveLink] 
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APPENDIX  D.  SURVEY  RESULTS 


Q2:  On  average,  how  often  to  do  you  log  into  ESSENCE? 


100% 

80% 

60% 

40% 

20% 

0% 


n-51 


Daily 


Weekly  (a  couple  Monthly  (a  couple  Greater  than 
times  per  week)  times  per  month)  monthly  (a  couple 

times  per  year) 


Never 


N=142 


Q3:  Do  you  receive  e-mail  alerts  from  ESSENCE? 


100% 

80% 


60% 


40% 

20% 

0% 


Yes 


No  I  didn’t  know  I  could 

receive  e-mail  alen 


N=142 
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Q4:  Do  you  log  into  ESSENCE  only  if  you  receive 
an  e-mail  alert? 


Yes  No 


N-82 


05:  Which  e-inail  alert(s)  do  you  receive? 

100% 

80% 
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Q6:  Please  explain  why  you  do  not  receive  e-mail  alerts. 

I  am  a  program  manager  and  do  not  need  alerts  at  my  level. 

I  have  not  set  up  the  system  to  send  me  alerts. 

I  check  essence  everyday  so  I  see  no  need  to  have  e-mail  alerts 

I  get  enough  e-mail  already.  Please  don't  e-mail  me.  I'll  check  ESSENCE  every  day. 

I  don't  use  ESSENCE  as  a  primary  alerting  tool,  rather  as  a  quick  look  at  trends  when  I'm  investigating  a 
particular  phenomenon. 

When  I  got  me  account  Essence  stated  they  do  not  send  out  e-mail  alerts 

Because  I  check  the  alerts  daily  when  I  am  at  work  so  I  see  the  alerts. 

Didn't  know  feature  was  available 

They  don't  show  up 

Have  only  received  system  maintenance  alerts. 

Fills  up  e-mail  Inbox-  Cumbersome 

Daily  log  in  dispenses  with  the  need  for  alerts.  If  you  are  looking  at  them,  you  don't  need  an  e-mail  about 
them. 

I  JUST  NEVER  HAVE.  THERE  ARE  MANY  OF  US  IN  THIS  OFFICE  THAT  HAVE  ACCESS  TO 
ESSENCE.  THEY  MIGHT.  ONE  OF  OUR  STAFF  CHECKS  ESSENCE  DAILY  FOR  REPORTABLE 
DISEASES. 

Forward  deployed  to  AFG  for  past  7  months. 

Just  never  signed  up  for  them  not  necessary  for  my  job. 

I  have  technicians  who  monitor  and  notify  me  if  we  have  issues. 

Did  not  sign  up  for  e-mail  alerts. 

I  work  at  the  public  health  center 

I  only  receive  notes  about  up  grades. 

I  do  not  think  I  ever  elected  to  receive  them. 

I  am  not  a  local  level  user.  Navy  wide  alerts  would  be  too  much.  Also,  when  have  looked  at  alerts,  do 
not  find  them  helpful  -  there  is  no  history  contained  in  the  alert. 

Did  not  know  I  could  nor  how  to  request  or  set  it  up 

Way  too  many  false  negatives  to  make  it  useful 

Have  not  requested  alerts 

I  have  not  requested  e-mail  alerts 

I  don't  know  why  I  don't  receive  e-mail  alerts. 

Other  people  w/in  the  PH  Directorate  receive  ESSENCE  e-mail  alerts 

Because  of  other  methods  of  information. 

I  rarely  use  essence  and  that  is  to  check  for  my  staff  members.  So  I  cannot  tell  you  why  I  don't  receive 
the  colored  alerts.  I  have  received  e-mails  stating  that  that  my  Essence  account  is  about  to  expire. 

IT  IS  NOT  MY  PROGRAM  ONLY  CHECK  WHEN  I  HAVE  THE  PREVENTIVE  MEDICINE  DUTY 
ONE  WEEK  A  MONTH 

I  am  not  sure,  The  only  alerts  I  receive  is  for  down  time 
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Q7:  Do  you  monitor  the  Alert  List  module? 


Q8:  Please  explain  why  you  do  not  use  the  Alert  List  to  monitor  alerts. 

I  look  at  the  e-mail  alerts,  log  into  essence,  and  look  for  trends. 

No  because  I  do  not  receive  the  alerts. 

No  idea  how 

I  am  in  Business  OPS  and  I  normally  get  into  ESSENCE  at  the  request  of  the  ESC  members 

I  just  returned  from  deployment  and  have  not  reactivated  my  account. 

Fills  inbox 

I  HAVE  NO  IDEAL  WHAT  IT  IS. 

I  am  an  Administrative  Clerk.  I  have  no  access  to  ESSENCE,  nor  will  I  be  using  the  program  to  monitor 
any  medical  info/updates. 

I  have  not  used  ESSENCE  in  over  a  year  since  I  left  the  Preventive  Medicine  field. 

Forward  deployed  to  AFG  for  past  7  months. 

It’s  a  passive  surveillance  system  and  there  is  no  authority  to  ensure  proper  reporting  or  accountability  of 
users  who  are  “supposed”  to  input  data.  Users  and  data  entry  personnel  are  not  properly  trained.  Even 
when  following  up  alerts  or  after  review  of  surveillance  data,  end  users  would  not  respond  to  inquiries  or 
even  know  anything  about  the  diseases  of  concern,  the  cases,  or  the  circumstances  of  the  interested 
diseases.  ESSENCE  and  the  process  in  which  is  it  currently  used  is  not  a  useful  program,  period! 

I  am  not  the  primary  user  that  the  command.  I  only  access  when  the  primary  user  is  on  leave. 

Not  part  of  my  job  description 

Our  Command  EHO  does  that,  we  only  cover  if  he  is  away  from  the  office 

The  alerts  are  e-mailed  to  me,  so  I  don’t  routinely  log  in  to  check  the  list. 

Ditto 

Other  people  use  it  and  inform  me 

I  already  receive  alerts  that  are  specific  to  me;  therefore  I  do  not  feel  the  need  to  always  monitor  the  alert 
list,  which  includes  other  facilities.  I  do,  however,  occasionally  look  at  it. 

Not  sure  what  you  mean  by  alert  lists.  I  use  alert  e-mails,  which  I  really  don’t  like  because  for  large 

AOR,  such  as  mine,  I  receive  so  many  alerts. 

I  don’t  have  access  to  many  features  of  ESSENCE  so  the  database  is  useless  to  me.  I  submitted  2  trouble 
tickets  so  far  and  my  issues  have  not  been  resolved. 
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Q8:  Please  explain  why  you  do  not  use  the  Alert  List  to  monitor  alerts. 


Don’t  know  how 

Not  the  primary  monitor  for  Essence  at  this  command.  Think  it  would  be  helpful,  though. _ 

We  have  an  HM1  assigned  to  do  monitor  ESSENCE  daily.  He  cuts/pastes  and  forwards  the  SADR 

MTF-Based  Temporal  Alerts  and  Reportable  Disease  list  to  the  chain  of  command  daily. _ 

I  have  someone  else  who  monitors  it.  I  will  do  occasional  query  for  a  disease  of  interest. _ 

I  am  not  one  of  the  primary  ESSENCE  monitors  at  this  duty  station  and  only  access  ESSENCE  when 

needed,  which  has  been  rare _ 

Co  workers  monitor  that  aspect _ 


Q9:  On  average,  how  often  do  you  run  a  reportable  diseases  Query? 


11=24  n.21  n=22 

■  ■  ■  ■  _ 

Daily  Weekly  (a  couple  Monthly  (a  couple  More  than  monthly  Never  I  don't  know  what 

times  per  week)  times  per  month!  (a  couple  times  per  this  is 

year) 
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Q10:  Do  you  have  a  Navy  Disease  Reporting  System, 
internet  (NDRSi)  account? 


100% 

80% 

60% 

40% 

20% 

0% 


n=89 


Qll:  If  confirmed,  do  you  or  do  you  with  a  co-worker  verify 
that  the  reportable  diseases  you  see  in  ESSENCE  are  also 
inputed  into  NDRSi? 

100% 


80% 


60% 


40% 


20% 


0% 


n=96 


n=12 


No 


N=108 
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Q12:  How  much  do  you  agree  or  disagree  with  the  following 
statement?  I  find  that  most  often  the  reportable  diseases  1  see  in 
ESSENCE  are  rule-outs. 

100%  "i — 

80% 


Q13:  Please  explain  why  you  do  not  use  the  reportable  diseases  Query  module. 

Did  not  know  this  was  available  on  this  application. 

Don't  use  essence  often  and  still  learning  essence 

This  is  done  by  our  diseases  folks  I  only  review  some  of  the  data  in  a  pinch 

I  did  in  the  past,  but  found  the  MTFs  in  our  area  are  familiar  with  it  and  use  it  as  a  check  against  MERs 
submitted. 

I  have  no  access  to  ESSENCE  program 

I  have  a  personnel  that  do  this  specifically  and  he  briefed  me  on  the  things  that  he  do 

I  haven't  used  ESSENCE  in  over  a  year  since  I  left  the  Preventive  Medicine  field. 

No  need 

It's  a  passive  surveillance  system  and  there  is  no  authority  to  ensure  proper  reporting  or  accountability  of 
users  who  are  “supposed”  to  input  data.  Users  and  data  entry  personnel  are  not  properly  trained.  Even 
when  following  up  alerts  or  after  review  of  surveillance  data,  end  users  would  not  respond  to  inquiries  or 
even  know  anything  about  the  diseases  of  concern,  the  cases,  or  the  circumstances  of  the  interested 
diseases.  ESSENCE  and  the  process  in  which  is  it  currently  used  is  not  a  useful  program,  period! 

I  am  not  a  Preventive  Medicine  Technician.  There  is  someone  in  the  department  designated  to  handle  the 
reporting  of  disease. 

I  have  other  ways  to  get  the  information. 

Not  trained  on  how. 

I  only  look  for  alerts  that  are  in  reference  to  my  area. 

Others  in  department  to  do  it. 

See  previous  comment  on  the  ongoing  issues 

Because  do  not  have  access  to  ESSENCE. 

Not  as  familiar  with  Essence  as  I  should.  Time  is  not  on  my  side  at  this  time. 

I  don't  know  how  too. 
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100% 


Q16:  When  you  run  syndromic  Query,  how  often  arc  the  following  parameters 
_  _  important  lo  >our  investigation?  _ 


Age  Range  Clinic  Type  Sum  Dale  F.iulDate  Paiiem  Category  Pat  em  Mil  inn  v 

Branch 


Never 
Ruidy 
“  SouicLiuca 

■  Frequently 

■  Always 


Q17:  For  a  syndromic  Query,  do  you  do  the  following  during  your  investigation? 


100% 


I  nanow  down  (01  .specify  J  |talicn1  I  look  at  syiuliomic  patterns  over  time  I  nvesligatc  ccitain  alciU  displayed  on 
paralleled  depending  on  tlie  syne  ionic/  the  data  I  aide  by  clicking  on  tltc  Data 

disease  I  am  Livcsligalnig  Details  fin  u  given  day 


N-89 


Q18:  Please  explain  why  you  do  not  use  the  syndromic  Query  module. 


Don’t  know.  Never  had  an  incident  to  use  it. _ 

If  there  is  an  issue  with  a  particular  syndrome  it  usually  shows  up  on  the  alert  list. _ 

Our  clinic  does  active  surveillance  of  infectious  diseases  including  acute  respiratory  disease  rates,  febrile 
respiratory  rates,  pneumonia,  soft  skin  tissue  infections,  conjunctivitis  and  many  others.  Our  data  is 
gathered  daily  or  the  next  day.  Essence  syndromic  query  is  usually  at  least  2-3  days  behind  the  current 
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Q18:  Please  explain  why  you  do  not  use  the  syndromic  Query  module. 

infection.  Our  surveillance  helps  us  explain  the  outcome  of  the  syndromic  queries  but  it  is  not  a  module 
we  use  on  a  regular  basis. 

We  do  daily  active  surveillance  of  syndromes  and  complaints  through  AHLTA  and  focus  on  positive 
results  for  reportable  diseases.  Where  we  get  our  information  almost  immediately,  ESSENCE  is  normally 
anywhere  from  a  few  days  to  a  week  behind  and  is  used  in  conjunction  as  a  secondary  check  to  our 
surveillance. 

I  have  personnel  that  specifically  do  this;  he  briefed  me  on  the  process. 

I  haven’t  used  ESSENCE  in  over  a  year  since  I  left  the  Preventive  Medicine  field. 

Not  familiar  with  it  or  how  to  use  the  data. 

We  look  at  Reportable  Diseases  only. 

No  need 

It’s  a  passive  surveillance  system  and  there  is  no  authority  to  ensure  proper  reporting  or  accountability  of 
users  who  are  “supposed”  to  input  data.  Users  and  data  entry  personnel  are  not  properly  trained.  Even 
when  following  up  alerts  or  after  review  of  surveillance  data,  end  users  would  not  respond  to  inquiries  or 
even  know  anything  about  the  diseases  of  concern,  the  cases,  or  the  circumstances  of  the  interested 
diseases.  ESSENCE  and  the  process  in  which  is  it  currently  used  is  not  a  useful  program,  period! 

I  do  not  see  this  as  a  value-added  service. 

Not  my  expertise. 

I  supervise  those  who  do 

Haven’t  been  trained  in  its  use. 

Ask  others  to  do  it. 

See  previous  comment  on  the  ongoing  issues 

Didn’t  really  understand  that  it  a  tool  available  until  this  survey. 

I  don’t  understand  what  it  is  for. 

No  need. 

I  don’t  have  the  need  for  it. 

I  don’t  know  how  too. 

Q19:  Do  you  investigate  alerts? 


100% 

80% 

60% 

40% 


20% 


0% 


Yes 


No 


N=136 


100 


100% 

80% 

60% 

40% 

20% 

0% 


Q20:  Do  you  use  ESSENCE  to  investigate  alerts? 


n=92 


Yes  No 


N=103 


Q21:  When  you  begin  an  investigation  using  ESSENCE,  which  of 
the  following  site  selections  do  you  choose? 


100%  T- 


Your  local  MTF  Only  Navy  and  All  services'  MTF’s  Across  all  Navy  and  Across  all  services’ 

Marine  Corps  in  your  region  Marine  Corps  MTFs  MTFs  in  your 
MTF's  in  your  in  your  country  country 

region 

N=92 
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Q22:  Once  you  notice  a  yellow  or  red  alert  for  a  particular 
syndrome,  do  you  look  up  data  details  within  ESSENCE  to  verify 
the  need  for  additional  investigation. 


Yes 


No 


N=84 
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Q24:  How  often  do  you  use  the  following  external  tools  to  further  investigate  clustering 

within  an  alert? 


100%  - — 


AHLTA  CHCS  Speak  with  providers  Speak  with  laboratory  Speak  with  immunizations 

technicians  technicians 


Never 

Kaiely 

■  Sometimes 

■  Frequently 

■  Always 


N=88 


Q24:  Other  (please  specify) 

Health  Level  7  data  at  NMCPHC 
Prev  Med  assets  @  MTF _ 

PREVENTIVE  MEDICINE _ 

Prev  Med  Tech's _ 

Local  PM  contact 

Preventive  Medicine  Techs  at  the  MTF 
Use  MRRS  and  Ahlta  for  imm  research. 


Q25:  Do  you  maintain  a  log  or  take  notes  (in  Word  or  Excel)  of  an 
investigated  alert  for  future  reference? 


investigated  and/or  the  severity  of 
the  outbreak  investigated 
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Q26:  After  researching  an  alert,  do  you  brief  your  command? 


100% 

80% 

60% 

40% 

20% 

0% 


n=70 

m 

i 

ii 

c 

n=6 

i  i  i 

Always  Depends  on  the  syndrome/  Never 

disease  investigated  and/or  the 

severity  of  the  outbreak 
investigated 


N=89 


Q27:  Please  explain  why  you  do  not  investigate  alerts. 

I  don’t  receive  alerts 

Depends  on  how  much  over  the  threshold  the  alert  is.  Sometimes  there  is  an  alert  for  one  or  two 
additional  patients...  Also  depends  on  the  syndrome 

I  haven’t  used  ESSENCE  in  over  a  year  since  I  left  the  Preventive  Medicine  field. 

I  use  this  as  an  aid,  but  investigate  the  reportable  diseases. 

Have  not  been  shown  this  portion  of  the  system  yet. 

No  need 

It’s  a  passive  surveillance  system  and  there  is  no  authority  to  ensure  proper  reporting  or  accountability  of 
users  who  are  “supposed”  to  input  data.  Users  and  data  entry  personnel  are  not  properly  trained.  Even 
when  following  up  alerts  or  after  review  of  surveillance  data,  end  users  would  not  respond  to  inquiries  or 
even  know  anything  about  the  diseases  of  concern,  the  cases,  or  the  circumstances  of  the  interested 
diseases.  ESSENCE  and  the  process  in  which  is  it  currently  used  is  not  a  useful  program,  period! 

Primary  users  usually  do  when  they  get  back 

Someone  else  is  designated  to  do  so. 

Although  the  alerts  can  point  out  an  ongoing  issue,  they  have  too  much  background  noise  to  be  of  use  as 
a  source  of  morbidity  surveillance/tracking. 

The  alerts  I  receive  are  not  reportable. 

Not  my  area  of  expertise. 

I  don’t  know  what  the  alert  list  is. 

See  above 

Haven’t  been  trained  in  this. 

My  AOR  that  I  receive  alerts  for  is  NME  and  NCA;  the  alerts  are  rarely  significant  because  they  account 
for  visits  over  those  large  areas. 

Just  now  getting  re-acclimated  to  system  for  use  at  the  unit  I  am  at. 
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Q27:  Please  explain  why  you  do  not  investigate  alerts. 


See  previous  comment  on  the  ongoing  issues _ 

Don’t  know  how _ 

Some  one  else  is  responsible  for  the  investigation.  I  just  oversee  that  it  is  being  completed  appropriately. 

I’m  not  sure  to  what  sort  of  alert  you  are  referring.  Do  you  mean  syndromic  flags? _ 

Never  seen  one _ 

I  only  run  reports.  Another  co-worker  does  the  investigating  and  reporting  verification. _ 

No  need _ 

I  do  not  receive  alerts. _ 

Co  workers  do  this  task 


Q28:  If  you  do  not  use  ESSENCE  to  investigate  an  alert,  what  alternate  steps  do  you  take  to 
conduct  your  investigation? 


Our  clinic  is  small  enough  for  me  to  walk  downstairs  and  ask  the  providers  about  an  increase  in  a 

syndrome. _ 

Patient/family/spouse  interviews _ 

Same  as  the  rest  of  the  questions  about  t  he  alerts _ 

I  contact  the  clinic  and  speak  with  the  clinic  manager  and  inquire  about  the  patients  for  that  day.  I  will 
also  look  in  ALTHA  and  read  the  notes,  check  labs  and  immunizations  records  and  demographics  to  see 

if  it  is  common  among  a  particular  population. _ 

We  use  AHLTA  notes  and  conversations  with  the  patient  and  provider. _ 

Conduct  patient  interviews,  utilize  ALTHA  notes  from  providers  and  laboratory  reports  if  necessary 

Look  over  patient  visits  in  AHLTA _ 

I  look  at  AHLTA,  CHCS  and  ER  data. _ 

Contact  command  if  possible _ 
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Q29:  Have  you  discovered  an  actual  outbreak  using  ESSENCE? 


100% 

80% 

60% 

40% 

20% 

0% 


n=99 


Yes 


No 


N=132 


Q30:  Check  all  the  outbreak  types  you  have  discovered  using 

ESSENCE. 

ioo%  - 


ILI  Heat  GI  Respiratory  Febrile  Neurological  Rash  None 


N=97 


Q30:  Other  (please  specify) 

Pertussis _ 

ESSENCE  is  used  but  is  normally  not  the  primary  for  the  use  in  an  investigation  (It  is  syndromic  sur) 
STD’s _ 

Exposure  to  chemical/toxin _ 

Animal  Bite 
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Q31:  Have  you  observed  a  miscoded  ICD-9  code  in  ESSENCE? 


100% 

80% 

60% 

40% 

20% 

0% 


n=100 


Q32:  What  steps  do  you  take  to  fix  miscodings?  Check  all  that 

apply. 


100% 

80% 

60% 

40% 

20% 

0% 


n=71 

1 

n=41 

n=33 

n=47 

■ 

■ 

M 

1  _ 

I  speak  with  the  I  speak  with  the  I  speak  with  the  MFT  I  speak  with  the  I  do  not  speak  with 
provider  immunizations  coding  representative  preventive  medicine  anyone 

representative  representative  at  the 

rlinir/hrKrifal 


N=99 


Q32:  Other  (please  specify) 

We  notify  NMCPHC  of  trends,  like  the  current  one  of  latent  TB  seen  as  pulmonary  TB  in 
ESSENCE... but  we  rarely  see  proof  of  anyone  taking  on  the  fix _ 

I  think  it  is  a  combination  of  people  where  the  error  occurs _ 

E-mail  the  Prev  med  contact  at  that  MTF _ 

I  speak  to  the  listed  ESSENCE  program  manager  at  the  MTF;  however,  this  person  is  not  always  a 
provider  or  trained  technician. _ 

I  have  observed  it  while  doing  OJT,  the  investigator  has  initiated  their  own  process _ 
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100% 

80% 

60% 

40% 

20% 

0% 


Q34:  How  long  did  it  take  you  to  acquire  your  current  ESSENCE 

account? 


n=51 


1-15  days  16-30  days  31-45  days  46  -  60  days  More  than  60  days  I  don't  remember 

N=132 


Q35:  Did  you  have  to  apply  more  than  once  to  obtain  your 
current  ESSENCE  account? 


100% 


80% 


60% 


40% 


20% 


0% 


n=82 


n=51 


Yes 


No 


N=133 
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Q37:  Estimate  how  many  times  you  needed  to  contact  the  helpdesk  (via  e-mail 
or  phone  call)  to  acquire  your  current  ESSENCE  account. 


Ill 


Q38:  Who  did  you  receive  your  ESSENCE  training  from? 
Check  all  that  apply. 


100% 


80% 


I 


My  predecessor 


My  boss 


I 


I  am  self-taught  I  downloaded  the  I  attended  a  conference  I  was  taught  by  the 

tSStNCb  training  where  I  was  taught  how  NfcPMU 

modules  and  tutorials  to  work  the  system 


Q38:  Other  (please  specify) 

Attended  training  at  OSDHA 

Sought  out  help  from  Army  Prev  Med  physician  and  drove  to  their  MTF  to  get  some  basic  training  on 
ESSENCE. 

I  have  no  essence  training 

No  one  I  have  asked  to  get  training  on  it 

I  really  had  no  formal  training-  there  is  an  SOP  on  our  share  drive.  I  read  it.  There  was  no  further 
guidance. 

I  have  been  using  it  for  about  7yrs 

Had  some  help  from  PMT 

Co-worker,  personnel  supervised 

No  training  was  given. 

Supervisor  &  self  taught 

Some  combination  of  them  (mainly  the  modules) 
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Q39:  Where  would  you  like  to  see  training  performed  for  current 
and  future  users  of  the  ESSENCE  system? 


100% 


At  my  own  desk  In  public  health  In  hospital/clinic  In  "C"  school 

conferences  meetings 

N=131 


Q39:  Other  (please  specify) 

I  think  consistent  training  is  necessary  to  ensure  that  we  are  all  looking  at  the  data  correctly  and  the  same 

There  should  be  a  training  course  held  somewhere  over  a  few  days  to  get  users  familiar  with  the  system. 
A  one  day  quick  run  through  via  something  like  NKO  would  be  a  complete  disaster. _ 

Preventive  Medicine  Technician  “C”  School,  NEPMUs _ 

At  MTF  facilities  so  that  other  MTF's  can  attend _ 

CME  training  on  site _ 

At  the  NEPMU _ 

Worksite _ 

PMT  C  school  for  essence  training _ 
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Q40:  How  would  you  like  to  see  training  provided  for  current 
and  future  users  of  the  ESSENCE  system? 


100% 


Audio  and  video  based  PowerPoint  presentations 


Hands-on  training 


N=131 


Q40:  Other  (please  specify) 


Webinar _ 

I  think  that  everyone  learns  differently  so  it  is  important  to  have  different  types  of  training 

Presentation  or  workshop  at  NMPHC  conference _ 

As  required _ 
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Q41:  What  type  of  training  would  you  like  for  current  and 
future  users  of  the  ESSENCE  system? 


Initial  training  to  Site  selection  set  How  to  evaluate  Query  module  Outbreak  scenarios 
use  ESSENCE  up  alerts  applications 

N=132 


Q41:  Other  (please  specify) 


How  to  detect  probable  false  positives _ 

See  above  response _ 

Use  of  the  Matrix  function _ 

Time-series  explanation,  14  day  incidence  explanation,  etc.  Explain  what  trigger's  alerts. 

Training/insight  into  how  syndromic  alerts  are  derived _ 


Q42:  Do  you  have  any  suggestions  that  will  help  make  the  ESSENCE  system  easier  to  use? 


Don’t  ask  for  me  to  change  my  password  as  often  as  requested.  I  have  been  on  leave  or  TAD  twice  when 
my  account  has  been  inactivated.  Although  I  should  have  completed  the  task  before  I  left,  I  don't  change 
my  bank  info  as  much  as  I  have  to  change  ESSENCE.  I  understand  the  privacy  issue,  but  most  medical 
folks  not  involved  in  Preventive  Medicine  wouldn't  even  know  where  to  access  ESSENCE  b/c  they  don't 
know  what  it  is.  I  think  it's  a  secure  site,  but  I  have  had  to  reset  my  password  several  times. _ 

None.  I  find  it  very  easy  to  use. _ 

-The  daily  requirement  to  check  ESSENCE  seems  to  be  a  bit  much.  I  find  that  many  times  the  alerts  are 
false  alerts  and  that  the  “syndrome”  parameters  are  so  wide  that  sometimes  the  alert  doesn't  seem 
credible,  (i.e.,  a  rash  in  2  infants  will  trigger  an  alert  and  so  will  ptosis  of  the  eyelid  diagnosed  in  the 
optometry  clinic.) 

-I  will  check  ESSENCE  and  then  check  the  AHLTA  encounter  and  contact  the  providers  or  clinics  to 
“ground  truth”  the  data  to  see  if  what  I  see  in  ESSENCE  is  truly  what  is  happening  in  the  clinics. 

-I  do  use  ESSENCE  to  see  how  we  are  doing  with  regard  to  trends  in  coding  for  certain  things  like  ILI, 
Resp,  and  GI  syndromes.  I  share  this  with  providers  as  appropriate.  I  also  check  the  reportable  disease 
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042:  Do  you  have  any  suggestions  that  will  help  make  the  ESSENCE  system  easier  to  use? 

section  and  use  this  as  a  “safety  net”  to  catch  any  reportable  diseases  that  weren’t  consulted  to  Prev  Med 
or  didn’t  show  up  as  a  positive  lab  in  our  box  in  the  Laboratory.  We  follow  up  on  each  reportable  disease 
that  shows  up  in  ESSENCE.  At  times  we  will  find  STDs  and  other  reportable  diseases  this  way. 
(Sometimes  we  find  that  they  are  simply  rule-outs  or  coding  errors,  i.e.,  “meningitis”  was  really  a  “part  2 
sea  duty  screen.”  Which  keeps  things  interesting.) 

Daily  log-ons  to  the  system  and  navigate  through  the  program  is  a  key  to  easier  use  and  familiarity. 

Understanding  how  the  alert  functions  work  would  be  better.  Right  now  I  feel  that  the  alerts  are  set  to  be 
too  sensitive,  as  we  get  quite  a  few  of  them  here.  Investigating  all  of  them  would  take  far  too  much 
time... when  we  do  investigate  them,  I  can  never  come  to  a  conclusion  on  the  cause  (if  any)  for  the 
increase  in  patients  for  that  particular  syndrome. 

Not  really,  it  is  relatively  easy  to  use.  It  is  not  very  easy  to  get  pt  info  even  with  Level  II  access. 

I  still  do  not  know  what  essence  is.  I  arrived  at  my  command  (first  duty  station)  and  I  was  slammed  with 
EIDS,  NDRSi,  ESSENCE,  AHLTA,  CHCS,  etc.  I  am  continually  baffled  as  to  why  the  Navy  wastes  so 
much  money  on  redundant  software  and  training.  Life  for  everyone  would  be  infinitely  simpler  if  one 
program  did  everything. 

I  think  that  the  time  that  it  takes  to  get  the  account  is  too  long.  Then  when  you  change  duty  stations  you 
have  to  change  your  DMIS  ID,  which  takes  time.  I  was  not  given  standardize  training  and  I  constantly 
have  staff  that  deploys  and  I  deploy  and  once  I  return  my  account  is  disabled.  Not  sure  why  we  are 
monitored  if  we  check  it  daily  since  most  of  the  information  is  not  updated  daily.  It  is  syndromic 
surveillance  so  it  is  not  an  active  system  you  still  have  to  run  lab  reports  daily  to  get  the  information  that 
you  need  for  reporting.  Too  many  systems.  Too  many  problems  with  the  system  and  it  is  not  even 
helpful.  The  average  Public  Health  person  will  already  know  that  they  have  an  outbreak  by  the  time  you 
see  anything  in  ESSENCE.  The  coding  is  another  problem,  especially  at  large  MTF  (e.g  NMCSD). 

Training  so  the  end  user  can  be  more  proficient.  And  have  alerts  sent  to  you  via  e-mail. 

Allow  us  to  drill  down  to  patient  level  data. 

Make  a  link  to  the  current  contact  at  the  prev  med  dept  of  the  MTF  from  which  the  case  was  seen  so  that 
is  a  one-click  process  to  make  an  inquiry  regarding  the  alert. 

I  would  suggest  better  training-  most  operational  IDC’s,  PMT's,  and  EHO’s  are  unaware  of  the  ESSENCE 
program. 

When  I  access  the  tool  it  requires  me  to  go  through  two  portals  EIDS  then  Essence.  Can  you  simply  this 
process 

Make  it  easier  to  view  patient  specific  information.  Currently  we  must  use  the  information  like  provider 
code,  date,  clinic  and  age  to  then  go  into  AHLTA  to  find  out  the  patients  name.  For  most  cases  the  name 
is  irrelevant,  however  it  is  important  to  distinguish  between  new  cases  and  f/u  cases  which  essence  does 
not  do,  and  for  identifying  reportable  cases.  Also  there  should  be  a  way  to  query  certain  ICD-9  codes  for 
specific  diseases,  currently  this  is  an  option  but  many  ICD-9  codes  are  not  identified.  It  would  be  great  if 
there  were  a  place  to  view  provider  codes,  as  many  of  these  are  not  representations  of  the  last  name  first 
name.  We  mostly  use  ESSENCE  to  identify  reportable  diseases  that  need  to  be  entered  into  NDRSI. 

Password  reset  is  a  major  pain.  I  spend  more  time  resetting  a  password,  one  because  of  frequency  and 
two  because  of  characters  and  length.  Every  has  to  write  it  down  to  remember  it  because  once  you  do  it 
changes. 

It  would  make  things  better  to  add  the  diagnosis  under  the  three  tier. 

No  the  process  to  get  access  is  sometimes  hard  within  the  commands  due  to  the  COC  and  approval 
routing. 

GET  PROPER  TRAINING  FROM  SOMEONE  WHO  KNOWS  HOW  TO  USE  IT. 

Access  to  historical  data  (run  through  detection  algorithms) 
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042:  Do  you  have  any  suggestions  that  will  help  make  the  ESSENCE  system  easier  to  use? 

meta-analysis  of  alerts 

Make  it  user  friendly 

I  counted  there  are  47  key  strokes/actions  required  to  login — many  steps  seem  redundant—  I  have  never 
used  a  program— to  include  high  level  SECRET  security  data  bases— that  requires  as  much  effort  to  sign 
in — and  hence  opportunities  for  mistakes.  Some  streamlining  of  the  sign-in  process  would  help  improve 
the  experience;  cut  down  on  non-value  added  time  spent,  and  increase  compliance  with  monitoring. 

-It  is  very  user-friendly 

Getting  access  to  the  system  should  be  easier.  Most  if  not  all  users  need  level  2  access  as  well. 

It  takes  too  much  time  to  track  down  the  cases  using  CHCS/AHLTA  with  just  a  clinic,  date,  and  ICD9. 
Most  of  the  folks  that  use  ESSENCE  have  access  to  all  of  the  PII  data  though  other  systems  and  I  don't 
understand  why  we  could  not  get  the  patient  identifiers  in  ESSENCE  to  save  time  investigating. 

A  problem  that  I  encountered  was  that  some  of  the  alerts  were  affected  by  follow  up  patients  therefore 
skewing  the  ACTUAL  numbers.  The  follow-ups  that  are  included  int  he  alert  numbers  thus  causing  a 
false  alert  within  the  system.  It  is  cumbersome  to  sit  there  and  sort  all  of  the  pins/dates/ICD-9  codes  and 
sort,  as  well  as  track  the  busy  provider  to  verify. 

Direct  access  patient  information  to  AHLTA/CHCS  and  NDRSi.  Too  much  time  is  spent  switching 
between  programs  to  find  patient  information  in  order  to  ensure  treatment  and  that  reporting  was 
completed. 

Quicker  uploads,  change  how  it  represents  a  red/yellow  warning  in  regards  to  .12  or  .2  or  why  is  less  than 
one  case  a  trend? 

I  lose  access  every  time  I  transfer  duty  stations  or  get  deployed.  Maybe  making  the  access  process  easier 
would  be  helpful 

1 .  Have  it  not  mix  follow-up  visits  with  initial  visits,  or  at  least  report  them  as  a  separate  category. 

Mixing  follow-ups  with  initial  presentations  inflates  case  counts. 

2.  Need  better  explanations  about  how  to  set  up  queries,  and  also  about  the  various  modules.  For 
example,  I've  never  been  able  to  get  the  map  function  to  work  correctly. 

Why  does  it  time  out  so  quickly?  When  accessing  multiple  cases  I  don't  have  enough  time  in  ESSENCE 
to  complete  all  the  tasks  before  it  times  out. 

Essence  is  definitely  a  great  tool  to  investigate  and  be  aware  of  certain  medical  concerns  in  your  area  of 
practice  however,  I  did  encounter  in  several  instances  that  information  provided  through  the  database  is 
minimal.  ICD  9  codes  inputted  in  the  system  are  sometimes  misleading  since  most  providers  use  a 
generalize  code  to  cover  rule-outs.  Also,  when  confirmed  through  ALTHA  or  CHCS,  the  results  often 
do  not  match  what  was  initially  entered  thus  making  all  initial  ESSENCE  entry  somewhat  useless. 

Essence  itself  is  a  good  tool  for  the  service  it  intends  to  provide.  But  the  biggest  problem  in  my 
experience  is  the  software  provider  that  supports  it.  It  is  not  user  friendly  for  those  persons  who  are 
trying  to  set  up  their  account.  It  has  been  stubborn  in  the  past  with  password  submission,  and  the  system 
has  had  nervous  breakdowns  jealous  fits  of  rage  if  you  accidently  put  in  the  wrong  password.  It  has  been 
unnecessarily  difficult  to  try  to  create  new  passwords  in  the  past.  A  savvy  user,  through  trial  and  error 
recognizes  that,  but  it  can  be  extremely  frustrating  for  a  new  user.  I  just  think  improvements  should  be 
made  to  improve  the  overall  system  and  make  it  much  more  user-friendly. 

Better  training  opportunities 

Streamline  the  establishment  of  accounts.  It  took  approximately  two  months  each  and  much 
correspondence  with  the  help  desk  for  each  of  two  PMT's  I  had  working  for  me  at  my  previous  command 
to  set  up  accounts. 

Grouping  -  it  would  be  helpful  to  group  all  entries  for  a  particular  patient's  diagnosis  together  to  avoid 
duplicate  reporting.  Often  checking  current  date  ESSENCE  and  discovering  a  follow-up  visit  for  a 
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patient  leads  to  reporting  that  patient  again  even  though  he/she  may  have  been  initially  seen  months 
before  for  same  event. 

Someway  to  filter  out  miscodes 

If  possible,  have  trainings  on  ESSENCE,  in  practical  application  settings. 

The  log  in  and  hard  to  use  Web  layout  makes  it  very  undesirable  to  use 

Improve  the  application  process,  which  includes:  removing  obsolete  application  forms,  combining  the 
requirements  with  the  other  databases,  such  as  CDM  so  that  items,  such  as  AI  training  does  not  have  to 
be  resubmitted,  have  person  doing  the  app  screening  ask  for  all  the  additional  info  needed  at  once  instead 
of  responding  (a  week  later)  after  one  item  has  been  submitted  then  asking  for  something  else. 

Fix  the  issues  that  I  have  had  since  finally  gaining  access  so  I  could  use  the  database. 

I’m  new  to  the  Essence  system,  therefore  very  limited  feedback  at  this  point.  I  was  prompted  by  several 
e-mails  to  complete  this  survey  though  I’ve  only  recently  begun  using  it.  Perhaps  another  e-mail  reminder 
in  6  months  would  be  more  useful. 

I  have  a  very  difficult  time  with  getting  access  to  patient’s  PHI.  I  have  called  several  times  and  still  don’t 
ever  have  access.  It  would  make  it  much  easier  if  they  would  stop  taking  away  this  access.  I  have 
completed  the  necessary  paper  work  and  it  was  approved,  it  is  just  a  systems  issue. 

User  defined  macros  (or  something  similar)  for  running  the  same  queries  day  after  day  (e.g.,  daily 
surveillance  tasks  at  the  NEPMUs. 

Need  hands  on  training,  time  management.  A  system  that  works  in  practical  terms  for  your  command. 

Provide  training  for  ESSENCE  in  “Hospital  Corpsman  A  school”  and  Preventive  Medicine  Technician 
“C”  school. 

Double  login  is  kind  of  a  pain.  Any  way  to  use  CAC  cards  for  at  least  one  or  both? 

I  use  ESSENCE  as  a  checks  and  balance  for  my  surveillance  for  reportable  communicable  diseases 
implemented  in  the  MTF.  I  use  CHCS  and  ALTHA  and  labs  as  my  surveillance.  It  is  somewhat  useful 
for  syndromes. 

Essence  is  a  great  program  but  timelines  could  use  a  little  more  finesse.  The  ICD9  codes  used  are 
sometimes  generalize  due  to  rule  outs.  Continuity  of  care  for  these  patients  are  logged  differently  thus 
making  the  Essence  user  have  to  dig  through  piles  on  information  just  to  confirm  that  it  is  a  change  in 
that  patients  status.  It  would  be  nice  to  be  able  to  categorize  per  patient  to  see  the  succession.  ALTHA 
and  CHCS  will  provide  this  information  obviously  but  it  would  have  been  nice  if  this  ability  is  available 
through  Essence. 

If  this  ability  is  already  programmed  through  Essence  then  I  apologize  for  this  comment  and  might  need 
to  look  through  the  training  materials  again. 

Better  training 

Improved  maneuverability  between  ESSENCE  and  NDRSi  (one  account  or  link  to  one  from  the  other). 
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Q43:  Do  you  have  any  suggestions  that  will  help  make  the  ESSENCE  system  more  effective? 


Password  convenience-  linked  to  my  CAC  card. _ 

Corrected  miscoding  would  really  help,  especially  when  it  is  obvious  that  a  MTF  has  miscoded  a  lot,  like 

immunizations  as  actual  disease. _ 

-I  think  if  the  coding  could  be  changed  in  ESSENCE,  that  would  be  helpful.  For  example  I  had  a 
provider  who  saw  a  “pustulosis”  in  a  pt  following  a  smallpox  vaccination,  and  coded  it  as  “Poisoning- 
Smallpox  vaccine”  When  we  discussed  it  with  the  provider,  they  had  no  idea  it  was  showing  up  in  the 
surveillance  data  this  way  and  even  when  they  changed  the  coding  in  the  AHLTA  encounter,  it  did  not 
update  ESSENCE. 

-I  really  had  minimal  training  in  how  to  use  ESSENCE  and  certainly  would've  appreciated  some  training 
especially  when  there  is  a  daily  requirement  to  check  it.  It  would  be  nice  to  know  what  am  I  expected  to 
do  if  I  see  something  in  ESSENCE?  And  how  do  I  know  it's  “real”?  We've  sort  of  figured  out  our  own 
way  to  vet  the  ESSENCE  data,  but  it  would  be  nice  to  know  what  was  intended/expected. 

-As  far  as  the  Chem  bio  syndromic  surveillance,  I  don't  know  how  this  could  be  that  useful  with  the  time 
delay  in  ESSENCE.  I  am  guessing  we  would  get  a  call  or  hear  through  the  grapevine  that  the  ER  or 
clinics  were  seeing  a  lot  of  patients  with  the  same  symptoms... but  would  be  surprised  if  we  actually  saw 
it  reflected  in  ESSENCE  until  a  day  later,  which  may  or  may  not  be  helpful. 

-I  do  like  being  able  to  compare  last  year's  ILI  visits  with  this  year.  Providers  liked  seeing  that  as  well. 

We  could  also  see  the  1st  and  2nd  wave  of  H1N1  last  year  and  that  was  interesting. _ 

Frequent  or  regular  training  for  people  who  feed  the  program/system  with  data.  This  will  reduce  some  of 

the  erroneous  entries,  i.e.,  miscodings  etc. _ 

Include  some  dx  that  are  specific  for  recruit  surveillance,  such  as  pneumonia  and  pharyngitis. _ 

I  like  the  map  function;  however,  when  your  entire  base  is  in  one  zip  code,  it's  pretty  useless.  If  there 

was  a  way  to  break  things  down  by  barracks,  that  would  be  better  (though  probably  not  feasible). _ 

At  Great  Lakes... ESSENCE  has  a  major  gap  ...it  does  not  show  VA  output  clinic  (ED)  data  for  AD 
members  seen  at  the  Federal  Health  Care  Center.  Essentially,  we  can  view  mainly  Brach  Health  Clinic 
output  data....  Many  recruits  and  students  visit  the  ED. and  weekend  info  is  also  gapped  when  ED  is  the 
venue  when  clinics  are  closed.  Since  the  federal  healthcare  center  uses  VA  systems  at  the  ED,  perhaps 
DOD  Essence  cannot  make  the  link....  I  have  communicated  this  concern  repeatedly  for  [redacted] 
years...NMCPHC,  ESSENCE  help  desk,  DHSS  Mr.  Aquaviv...  but  no  closure.  Anyway,  since  you 

asked,  this  is  important  to  us.  [redacted] _ 

Get  rid  of  it.  Have  a  program  that  is  used  by  most  of  the  medical  community  (CHCS  or  AHLTA)  do 
disease  reporting.  When  we  do  reports  here  the  information  we  need  we  have  to  get  from 
AHLTA/CHCS  so  why  not  just  get  rid  of  the  middleman  and  have  those  programs  report  the  same  things 

as  ESSENCE. _ 

I  like  the  mapping  function  and  the  fact  that  you  can  put  it  into  a  spreadsheet  but  those  are  the  only  useful 
applications  of  the  program.  I  would  get  rid  of  this  program.  We  already  do  active  lab  surveillance,  why 

create  more  work  for  everyone? _ 

Correct  issues  with  coding  errors  if  possible  and  it  would  be  nice  if  repeat  diagnosises  could  be  ruled  out. 
For  example  if  we  have  a  case  of  Lymes  and  it  is  the  initial  case  and  it  is  reported  in  NDRSi  and  locally 
every  time  that  patient  comes  back  for  follow  he  shows  up  on  the  reportable  list  even  though  he  has 

already  been  reported. _ 

Allow  miscodes  to  be  corrected  and  reflected  in  ESSENCE.  For  some  reason,  latent  TB  has  been  coded 

recently  as  pulmonary  tuberculosis,  NEC.  Really  misleading _ 

Get  it  to  talk  to  AHLTA  in  such  a  way  that  we  can  pull  up  an  AHLTA  note  from  the  alert  case-  eg  that 
way  we  could  tell  ourselves  that  case  was  miscoded  as  TB  when  it  is  really  LTBI.  We  spend  too  much 
time  spinning  our  wheels  chasing  wild  geese  with  miscodes  that  when  an  actual  outbreak  were  to  occur 
no  one  will  pay  attention-  like  the  boy  who  cried  wolf  too  many  times. _ 


043:  Do  you  have  any  suggestions  that  will  help  make  the  ESSENCE  system  more  effective? 

SADR  entries  (DX  codes)  will  never  be  100  percent  accurate,  providers  enter  AHLTA  dx  to  the  best  of 
their  ability  but  the  myriad  of  DX  options  in  AHLTA  can  make  exact  dx  difficult  at  times.  It  is 
incompact  upon  the  essence  review-to-review  clinical  cases  in  AHLTA  to  determine  relevance  (not  the 
role  of  the  primary  care  provider  to  give  the  dx  that  the  ESSENCE  user  wants). 

Make  it  easier  to  view  patient  specific  information.  Currently  we  must  use  the  information  like  provider 
code,  date,  clinic  and  age  to  then  go  into  AHLTA  to  find  out  the  patients  name.  For  most  cases  the  name 
is  irrelevant,  however  it  is  important  to  distinguish  between  new  cases  and  f/u  cases,  which  essence  does 
not  do,  and  for  identifying  reportable  cases.  Also  there  should  be  a  way  to  query  certain  ICD-9  codes  for 
specific  diseases,  currently  this  is  an  option  but  many  ICD-9  codes  are  not  identified.  It  would  be  great  if 
there  were  a  place  to  view  provider  codes  as  many  of  these  are  not  representations  of  the  last  name  first 
name.  We  mostly  use  ESSENCE  to  identify  reportable  diseases  that  need  to  be  entered  into  NDRSI. 

I  understand  that  there  are  a  lot  of  factors  to  take  into  consideration  but  If  there  were  a  way  to  have  the 
information  updated  to  show  more  of  a  real  time  snapshot  of  what  is  going  on  instead  of  the  delay  in 
reporting  I  think  it  would  help  to  get  more  of  a  control  on  outbreaks  sooner  than  later  for  those  that  use 
ESSENCE  only  for  tracking. 

I  also  think  that  having  some  type  of  imbedded  provider  index  listing  for  the  provider  codes  would 
alleviate  the  waste  in  time  that  is  used  to  track  down,  which  provider  entered  data. 

Allow  the  user  the  ability  to  cut  and  paste  graphs  onto  a  word  document.  A  feature  that  was  lost  roughly 
a  year  ago. 

BE  ABLE  TO  ACCESS  IT  VIA  CAC. 

Discussion  board  tied  to  alerts,  so  that  investigated  alerts  are  tagged  and  results  are  known 

Keep  it  simple 

Allow  only  one  diagnosis  per  visit.  The  idea  of  the  final  diagnosis  is  good. 

Training... 

Not  all  the  encounters  have  the  patient  information  available  even  though  you  have  access  with  Level  II. 

-Provide  an  accessible  PIN  for  cases  on  the  reportable  disease  query 
-Ensure  the  graph  populates  when  using  the  matrix 
-Provide  more  spaces  for  the  physician's  full  last  name  and  initials 
-14day  incidence  can  be  confusing 

-Better  training  programs  need  to  be  implemented. 

-Patient  identifiers  need  to  be  included  in  the  reportable  event  module. 

Identify  a  way  to  report/track  follow-ups  versus  new  encounters. 

New  user,  not  sure  at  this  time. 

The  real  problem  with  ESSENCE  is  not  necessarily  the  IT  system  itself,  though  it's  not  perfect,  it's  the 
overall  health  system  in  which  it  is  employed.  We  need  trained  users.  We  need  the  users  to  be  PM/PH 
trained,  not  some  nurse  or  EHO  who  has  it  as  a  collateral  duty.  NMCPHC  and  NEPMU's  need  authority 
to  both  assist  and  ensure  accountability  to  MTF's  to  report.  We  need  recurring  ESSENCE  and  disease 
reporting  training  at  the  MTF  and  smaller  unit  levels.  With  strengths  in  these  areas,  then  we  can  trust  the 
data  ESSENCE  provides  and  ensure  timely  input-feedback  to  actually  make  a  difference  in  prevention. 

Fix  the  system  not  the  IT  program! ! 

Clean  up  the  contact  PIN  to  be  more  direct. 

I  do  not  think  this  system  is  valuable  at  the  MTF  level.  Its  use  should  be  de-emphasized  below  the  level 
of  the  NEPMU/NMCPHC.  It  is  useful  for  trend  analysis,  but  not  real-time  investigation  (except  to  note 
general  trends). 

Overlaps  with  above  response. 

No,  it's  a  very  good  system;  it's  just  the  getting  an  initial  account  can  take  a  long  time. 
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043:  Do  you  have  any  suggestions  that  will  help  make  the  ESSENCE  system  more  effective? 

If  you  had  a  “pop  up”  in  AHLTA  that  read  “are  you  sure  you  want  to  code  ....etc..”  It  may  reduce  the 
false  positives. 

It  would  be  nice  if  ALTHA  and  or  CHCS  communicate  with  ESSENCE.  This  would  provide  a  better 
picture  of  the  true  nature  of  all  medical  concerns  in  the  area.  If  results  haven’t  been  verified  as  true  or 
positive,  those  entries  shouldn’t  be  entered  into  the  system. 

-In  using  the  system  in  conjunction  with  NDRSi,  It  would  be  helpful  if  NDRSi  search  field  was  by  date 
of  report  and  not  by  date  of  onset. 

-Training  on  how  to  use  various  modules  to  the  system  i.e.,  for  syndromic  surveillance,  how  are  the 
pharmacy,  radiology  modules  used? 

As  I  understand  Essence  it  pulls  data  straight  from  ALTHA/CHCS  based  on  diagnoses  code.  If  the 
diagnoses  are  incorrect  or  the  lab  results  are  negative  how  can  this  be  corrected.  I  have  instances  when 
submission  is  incorrect  but  the  data  could  not  be  edited. 

Make  it  easier  to  get  accounts  and  log  in.  Make  it  a  more  user-friendly  interface. 

As  stated  in  your  survey,  the  removal  of  repeat  patients  and  the  miscodes  would  make  the  system  more 
effective. 

Local  prev  med  contact  info  for  locations,  available  via  clicking  on  a  location  in  a  line  item,  perhaps. 

Weekly  hands  on  training.  Don’t  use  it  forget  about  it.  Weekly  or  bi-weekly  training. 

Ensure  that  “all”  demographic  information  on  a  patient  is  entered  in  the  case  that  an  investigation  is 
needed. 

Password  should  be  required  to  change  less  frequently. 

It  needs  to  work  with  CHCS  and  ALTHA. 

As  I  said,  it  is  a  great  program  but  could  really  use  an  easier  interface  for  those  who  haven’t  used  it 
before.  The  PPT  presentations  and  Audio  visual  trainings  are  somewhat  ok  but  only  talks  about  the 
basics.  It  would  be  nice  to  incorporate  tips  on  how  to  utilize  the  system  efficiently. 

If  it  was  closer  to  “real  time”  that  would  be  great.  ESSENCE  seems  to  be  about  5-10  days  behind 

Tutorial  explaining  how  ESSENCE  gets  the  data  it  reports. 

Q44:  Do  you  have  any  other  comments  that  you  would  like  to  make  regarding  the  ESSENCE 
system? 


Great  system. _ 

I  would  like  to  see  a  faster  turnaround  from  when  a  case  is  reported  into  CHCS/AHLTA  to  when  it 

appears  in  ESSENCE. _ 

Too  sensitive.  One  GI  case  isn’t  an  outbreak  even  if  a  clinic  has  not  had  one  for  a  few  weeks. 

Need  denominator  data.  No  way  to  consider  rates  without  it. 

Too  sensitive.  If  I  investigated  every  alert  I  would  do  nothing  else. 

Don’t  like  the  addition  of  influenza  specific.  I  need  to  query  flu  and  ILI  in  the  same  query _ 

Overall,  I  am  glad  that  ESSENCE  exists.  The  ability  to  perform  active  surveillance  is  wonderful,  as 
providers,  historically,  do  a  horrible  job  reporting  infectious  diseases  to  the  PM  A. 

I  am  still  on  the  fence  about  the  usefulness  of  the  alert  modules,  however. _ 
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Q44:  Do  you  have  any  other  comments  that  you  would  like  to  make  regarding  the  ESSENCE 
system? 

Training  needs  to  be  provided  at  the  “C”  school  level.  Students  that  come  from  PMT  “C”  school  today 
are  clueless  about  ESSENCE  and  we  have  to  spend  lots  of  time  training  them  on  its  use.  Also,  those  of 
us  who  have  had  to  learn  ESSENCE  on  the  job,  could  use  some  better  form  of  instruction  than  a  PP 
presentation. 

I  would  recommend  retreat  from  the  mandate  to  access  daily,  and  the  report  card  to  COs. 

Weekly  would  be  better.... 

It’s  PREVENTIVE  not  PREVENTATIVE 

Make  it  easier  for  Preventive  Medicine  Technicians  to  get  access  to  patient  identification  for  reportable 
diseases. 

Daily  checks  are  overkill,  alerts  fill  inboxes...  but  I  must  admit  in  the  end,  the  ESSENCE  system  is  worth 
having  especially  during  flu  season  and  the  last  H1N1  crisis. 

I  no  longer  use  Essence  due  to  my  job  change;  I  did  use  it  as  infection  control  officer  and  found  it  very 
useful  and  at  times  extremely  helpful  when  outbreaks  were  noted. 

It  has  been  a  great  additional  tool  to  capture  reportable  diseases  and  track  for  outbreak  trending.  Hope  it 
is  a  program  that  Navy  Medicine  and  the  Tri-Services  keep  as  our  Preventive  Medicine  Departments 
keep  getting  smaller  with  continued  deployment  support  missions. 

Too  many  passwords  to  remember... make  it  part  of  ALTHA 

As  the  survey  suggests,  there  are  many  false  positives... and  many  of  the  abbreviations  remain  a  mystery— 

-  a  key  or  legend  would  help — or  if  it  exists — make  it  more  apparent  to  access.  The  idea  of  the  program 
is  marvelous—  so  if  it  was  a  little  easier  to  use  I  think  more  of  us  would  use  it  as  it  was  intended. 

Thanks  for  asking. 

Excellent  surveillance  tools  if  you  have  direct  access  to  medical  records  like  AHLTA  or  CHCS. 

The  customer  service  is  excellent.  However,  we  do  not  use  NDRSi  to  confirm  ESSENCE.  We  use 
ESSENCE  to  input  new  cases  into  NDRSi  and  AHLTA  to  confirm  ESSENCE.  When  there  is  no 
accessible  PIN,  we  contact  a  POC  at  the  location  of  the  MTF  to  determine  the  identification  of  the  case  in 
question.  That  is  tedious  and  time  consuming. 

The  ESSENCE  program  is  good  as  long  as  the  users  understand  the  limitations  of  the  data.  Provider 
coding  is  what  feeds  the  machine  and  results  in  the  majority  of  the  issues  I  have. 

The  information  provided  by  me  in  this  survey  was  based  off  of  my  last  use  of  the  system.  I  used  the 
system  very  frequently  for  3  years  and  it  was  very  beneficial  in  identifying  and  preventing  outbreaks  in  a 
recruit  population,  which  allowed  us  to  intervene  and  take  appropriate  measures  to  minimize  illnesses 
and  diseases  in  a  timely  manner.  I  have  not  used  ESSENCE  in  over  a  year  due  to  my  leaving  the 

Preventive  Medicine  field. 

Still  trying  to  figure  it  all  out. 

Even  though  I  am  required  to  DOD  IA  required  training  at  the  hospital  I  worked  at,  I  still  needed  to 
complete  it  online  through  EIDS.  Double  efforts.  It  should  be  tied  in. 

Lengthen  the  lock  out  periods.  Supervisors  have  to  get  reloaded  if  we  don’t  review  it  frequently. 

Great  concept,  but  still  needs  some  improvement. 

ESSENCE  was  not  useful  during  H1N1  outbreak  last  year.  It  was  sometimes  more  than  48  hrs.  old  when 

I  saw  a  coded  case. 

Probably  takes  more  time  than  should  to  learn  to  navigate  EIDS/ESSENCE. 

I  would  definitely  like  to  have  had  more  training  on  using  the  system.  I  was  taught  by  a  co-worker  who 
was  a  new  tech  with  limited  experience  and  also  by  just  playing  around  with  the  system.  Some  training 
in  “C”  school  would  be  very  helpful;  also  training  at  the  annual  conference  by  the  personnel  at  the  Public 
Health  Center  would  be  helpful. 

Please  keep  upgrading  the  system  to  better  it.  Have  a  nice  day. 
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Q44:  Do  you  have  any  other  comments  that  you  would  like  to  make  regarding  the  ESSENCE 
system? 


Need  to  be  able  to  distinguish  between  TB  and  LTBI.  Coding  has  changed  somehow  and  when  provider 

codes  for  LTBI  it  comes  up  TB.  And  then  unable  to  correct. _ 

Training  modules  were  easy  to  use. _ 

I  think  the  system  is  a  good  way  for  surveillance  would  hate  to  see  it  done  away  with.  We  just  need 

hands  on  training.  More  confidence  to  navigate  through  the  system. _ 

I  like  the  program.  Communicable  diseases  (DNBI)  monitoring  and  controlling  through  ESSENCE  is  a 
breeze  once  you  get  the  hang  of  it.  Even  though  I  have  PCS'd  and  no  longer  access  the  program 
anymore,  I  still  recommend  training  for  all  corpsman  and  PMTs  as  part  of  any  annual  training  since  the 
program  can  be  used  in  conjunction  with  CHCS  and  ALTHA  to  pinpoint  on  any  disease  of  interest,  and 
allows  for  better  communication  to  the  axillary  departments  (LAB,  RAD,  PHAM,  PREVMED,  etc.)  If 

there  were  a  way  to  link  it  to  NDRSi,  this  program  would  be  perfect! _ 

The  DoD  disease  surveillance  process  can  be  streamlined  by  incorporating  capabilities  of  ESSENCE 
w/in  AHLTA  &/or  CHCS.  For  now,  we  expend  excessive  man-hours  volleying  between  ESSENCE, 
AHLTA,  CHCS,  ESSENTRIS  (ER  &  wards),  and  NDRSi.  NEPMUs  should  have  direct  access  to  this 
entire  software  series  to  view  all  patient  activities  w/in  all  Naval  UICs.  Many  Prev  Med  Depts.  simply 

do  not  have  the  man-hours  available  to  capitalize  on  the  sophisticated  epi  features  w/in  ESSENCE _ 

I  do  not  use  it  much.  I  “filled  in”  for  a  couple  of  weeks  and  occasionally  thereafter  for  the  person  who 

normally  runs  the  reports. _ 

There  need  to  be  some  form  of  formal  training  at  the  clinic  level. _ 
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Q45:  Select  the  current  user  level  that  best  describes  you. 


100.0% 

80.0% 

60.0% 

40.0% 

20.0% 

0.0% 


Q45:  Other  (please  specify) 

Corporate  level _ 

NHRC— all  recruit  training  camps _ 

No  longer  using  ESSENCE  in  current  navy  job 
Do  not  use. _ 

Nay- wide  (program  level) _ 

BUMED _ 

Haven't  use  it  since  March  2010 
Not  sure 
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Q46:  Select  the  facility  that  you  currently  support. 

80.0% 

60.0% 

40.0% 

20.0% 

0.0% 

N=120 


n=b0 


Clinic  Naval  Hospital  Naval  Medical  Center  NEPMU 


Q46:  Other  (please  specify) 

Navy  and  Marine  Corps  world-wide 

Naval  Hospital  and  12  branch  clinics 

James  A  Lovell  Federal  Health  Care  Center 

NHRC— all  recruit  training  camps 

Support  a  hospital  and  2  outlying  clinics 

R&D 

All  Navy  MTFs 

And  the  clinic  as  well 

BUMED  Navy  Medicine 

I  am  stationed  at  the  NEPMU,  but  doesn't  work  in  the  ESSENCE  department. 

FHCC 

125 


100.0% 

80.0% 

60.0% 

40.0% 

20.0% 

0.0% 


Q47:  Check  all  the  service(s)  you  currently  support 

n=131 


U.S.  Air  Force  U.S.  Army  U.S.  Marine  Corps  U.S.  Navy 

N=131 


Q48:  Select  the  professional  background  (hat  best  describes  you. 


Technician  Officer  Officers 


N=121 


Q48:  Other  (please  specify) 

Environmental  Health  Tech _ 

Infection  Control  Practitioner 
Public  Health  Admin _ 

Environmental  Health  Technician 
Administration  -  Business  OPS 
HCA,  Clinic  Head _ 

Program  Analyst  MPH _ 
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Q49:  Which  category  do  you  fall  into? 

1  uu.u/o 

QA  no/ 

60.0% 

n=50 

40.0% 

20  0% 

n=21  n=25  n=22 

U.UTo 

El  -  E3  E4  -  E6  E7  -  E9  Ol  -  03  04  -  06  Civilian  Contractor 

N=131 
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